Understanding facial recognition algorithms

Artificial neural networks are the most popular and successful method in image recognition. Facial recognition algorithms are based on mathematical calculations, and neural networks perform large numbers of mathematical operations simultaneously.

The algorithms perform three main tasks: detect faces in an image, video, or real-time stream; calculate a mathematical model of a face; compare models to training sets or databases to identify or verify a person.

This article covers the most well-known face recognition algorithms and key features. Since each method has its task-specific advantages, researchers actively experiment with combining methods and developing new technologies.

Table of Contents

Convolutional Neural Network (CNN)
Eigenfaces
Fisherfaces
Kernel Methods: PCA and SVM
Haar Cascades
Three-Dimensional Recognition
Skin Texture Analysis
Thermal Cameras
ANFIS
Local Binary Patterns Histograms (LBPH)
FaceNet
NEC
Megvii (Face++)
Facial Recognition: Combining Different Techniques
Summary
Facial Recognition Algorithms FAQs
How does the face detection algorithm work?
What is a facial recognition algorithm?
How can you do face detection in OpenCV?
How do you train facial recognition algorithms?

Convolutional Neural Network (CNN)

Convolutional neural network (CNN) is one of the breakthroughs of artificial neural networks (ANN) and AI development. It’s one of the most popular algorithms in deep learning, a type of machine learning in which a model learns to perform classification tasks directly on an image, video, text, or sound. The model shows impressive results in several fields: computer vision, natural language processing (NLP), and the largest image classification data set (Image Net). CNN is a normal neural network with new layers — convolutional and pooling. CNN can have dozens and hundreds of these layers, and each of them learns to detect different imaging features.

Eigenfaces

Eigenfaces is a face detection and recognition method that determines face variance in image data sets. It uses these variances to encode and decode faces with machine learning. A set of eigenfaces is a collection of “standardized face ingredients” determined by statistical analysis of a large number of face images. Facial features are assigned mathematical values, as this method doesn’t use digital pictures but rather statistical databases. Any human face is a combination of these values with different percentages.

Fisherfaces

Fisherfaces is one of the most popular facial recognition algorithms; it’s considered superior to many of its alternatives. As an improvement to the Eeigenfaces algorithm, it’s often compared to Eigenfaces and considered more successful in class distinction in the training process. The key advantage of this algorithm is its ability to interpolate and extrapolate over lighting and facial expression variation. There are reports of 93% accuracy of the Fisherfaces algorithm when combined with the PCA method at the preprocessing stage.

Kernel Methods: PCA and SVM

The principal component analysis (PCA) is a universal statistical method with many practical applications. When used in the face recognition process, PCA aims to reduce the source data size while preserving the most relevant information. It generates a set of weighted eigenvectors that, in their turn, build up eigenfaces — extensive sets of different human face images. A linear combination of eigenfaces represents every image in the training set. The PCA is used to receive these eigenvectors from the covariance matrix of a training image set. For each image, its main components are calculated (from 5 to 200). The other components encode minor differences between faces and noise. The recognition process includes comparing the unknown image’s main component to the components of all other images.

Support vector machine (SVM) is a machine learning algorithm that uses a two-group classification principle for distinguishing faces from “not-faces.” For each category, an SVM model receives a labeled training data set to categorize new test data. Researchers apply linear and nonlinear SVM training models for face recognition. The recent results show that the nonlinear training machine has a larger margin and better recognition and classification results.

Haar Cascades

Haar Cascade is an object detection method used to locate objects on images. The algorithm learns from a large number of positive and negative samples — the former contains an object of interest, and the latter contains anything other than the object you are looking for. After training, the classifier can find an object of interest on new images. The method was used in criminal identification in combination with the local binary pattern algorithm to recognize faces. The Haar cascade classifier uses 200 (out of 6000) features, which ensures an 85-95% recognition rate even with varying expressions.

Three-Dimensional Recognition

The underlying idea of 3D face recognition technology is the human skull’s unique structure. Each person’s skull structure is unique and can be described by several dozen parameters. This facial recognition method is based on comparing a 3D facial scan to the database patterns. It has an essential advantage — makeup, facial hair, glasses, and similar factors don’t affect the detection and recognition process. The latest research has used the technology of mapping the 3D geometry information on a regular 2D grid. It allows the combination of 3D data’s descriptiveness with 2D data’s computational efficiency and shows the highest performance reported on the FRGC v2 (Face Recognition Grand Challenge 3D facial database).

Skin Texture Analysis

Skin recognition technology has many applications — face detection algorithms, objectionable image filtering, hand gesture analysis, etc. It usually uses high-resolution images. Particular cases of skin texture analysis use different unique parameters like moles, skin color, skin tones, and many others. Recent research based on a combination of texture features and skin color showed interesting results. The researchers used a neural network to develop and test a skin recognition system. The feed-forward neural networks used in the project classified input texture images as “skin” and “non-skin” and showed an impressive performance.

Thermal Cameras

A thermal camera is a device used for monitoring the temperature distribution of the examined surface. The temperature distribution is displayed as a colored picture with different colors corresponding to temperatures. The technology already has several practical applications adapting to global changes — smartphone-based immunity certificates, remote fever detection, and thermal facial recognition. Thermal face recognition models are based on the unique temperature patterns of a human face. Human consistent temperature “signatures” are measured with thermal infrared (IR) imaginary. Using the thermal method in face recognition has an undeniable benefit — makeup, facial hair, hats, and glasses don’t affect its accuracy. Moreover, it distinguishes twin siblings.

ANFIS

An adaptive neuro-fuzzy interference system (ANFIS) is a type of artificial neural network. This method integrates the principles of neural networks with fuzzy logic principles and combines their advantages in a single structure. ANFIS is used to classify image features extracted from datasets on the preprocessing stage. Data scientists combine this method with a variety of feature extraction algorithms. Thus, some studies reported incredible 97.1% ANFIS classification accuracy after feature extraction with 2D principle component analysis.

Local Binary Patterns Histograms (LBPH)

This method uses local binary patterns (LBP), a simple, effective texture operator in computer vision that marks pixels in an image by setting each pixel’s neighborhood threshold and treating the result as a binary number. At the learning stage, the LBPH algorithm creates histograms for each image that is labeled and classified. Each histogram represents each image from the training set. This way, the actual recognition process implies comparing histograms of any two images.

FaceNet

The face recognition system FaceNet, developed by Google researchers in 2015, is based on face recognition benchmark datasets. Available pre-trained models and various open-source third-party implementations make this system quite wide-spread. FaceNet shows excellent results in research-conducting surveys, testing performance, and accuracy compared to other algorithms developed earlier. FaceNet accurately extracts face embeddings, high-quality features used for training face identification systems at later stages.

NEC

The solution developed by the Japanese technology company NEC allows highly accurate identification of people while recognizing age changes. The solution uses Adaptive Region Mixed Matching, a model that focuses on highly similar segments for mapping. The NEC technology divides input and registered images into small segments and focuses only on greater similarity segments. It allows the system to show higher identification accuracy, even in the case of the face wearing a mask or glasses. As its underlying algorithm, the NEC solution uses generalized learning vector quantization (GLVQ).

Megvii (Face++)

Chinese technology company Megvii has become known worldwide after introducing its face recognition application platform. It’s multi-purpose software. The Megvii algorithm is based on graph detection and fuzzy image search technology. The tech solution uses the company’s proprietary deep learning framework MegEngine, built on big data. The company’s technology successfully performs facial information extraction and includes several key features: face and human detection and tracking, face recognition and clustering, key-point detection, face attribute estimation, and face search engine.

Facial Recognition: Combining Different Techniques

Each facial recognition technology has its effective features. However, recent research has proved that the best results are achieved with combinations of different algorithms and methods. These combinations aim to solve the facial recognition process’s many routine problems — differences in facial expressions, posing, lighting conditions, image noise, etc. The latest experiments combine the LBP algorithm with advanced image processing techniques: bilateral filter, histogram equalization, contrast adjustment, and image blending. Such a technique shows impressive improvement to the LBP code and looks very promising for further research.

Summary

There is a variety of face recognition algorithms and methods. Although they all have one predominant aim, they can be task- and problem-specific. Depending on purposes of use and implementation circumstances, they range from neural networks and mathematical models to private companies’ tech solutions.

This article covered the most wide-spread algorithms and methods. However, more research and scientific experiments show the undeniable benefits of combining different algorithms to get better results in the facial recognition process. It results in the emergence of new techniques and use-specific methods.

Facial Recognition Algorithms FAQs

How does the face detection algorithm work?

Typically, specific neural networks are trained to detect human face landmarks and distinguish faces from other objects in the image. The landmarks are universal human facial features like eyes, nose, mouth, eyebrows, etc. The ways of practical implementation differ depending on the algorithm.

What is a facial recognition algorithm?

The facial recognition algorithm is a method of building a biometric face model for further analysis and the face recognition process.

How can you do face detection in OpenCV?

Open Source Computer Vision Library (OpenCV) is a popular library of computer vision algorithms, image processing, and numerical open-source general-purpose algorithms. Using OpenCV, you can carry out the face recognition process in three steps:

• Face detection;
• Feature extraction using face embedding;
• Facial recognition.

How do you train facial recognition algorithms?

Before facial recognition algorithms are ready for performing necessary tasks, they need to go through large amounts of data — precisely labeled image sets. These sets are used to develop a machine learning model.