How Accurate is Facial Recognition Today?

The viral spread of FRT has made it a hot topic and a controversial issue. Is facial recognition meant to make our lives safer, or does it expose us to additional risks? How accurate is FRT nowadays? Check out our comprehensive guide to find out!

What is Facial Recognition?
How Facial Recognition Works?
How Accurate is Facial Recognition?
Factors Affecting Face Recognition Accuracy
Aging
Facial Expression
Plastic Surgery
Occlusion
Low Resolution
Noise
Illumination
Pose Variation
How Has Facial Recognition Accuracy Improved?
How is Facial Recognition Accuracy Measured?
Why is a Threshold Important?
Is Face Recognition More Accurate Than The Human Eye?
Risks Faced with Facial Recognition
Identification errors
Confidentiality
Data Abuse
Summary
Face Recognition Accuracy FAQ
How can you improve the accuracy of face recognition?
Is facial recognition accurate for ID verification purposes?
Can face recognition be fooled?
How to calculate face recognition accuracy?
Why does face recognition accuracy vary due to race?
Recommended reading

What is Facial Recognition?

Face recognition is one of Computer Vision Technology’s capabilities based on algorithms known as neural networks. This technology aims to find, recognize, and distinguish faces. For such systems, an image is given a dataset with distinctive features. FRT algorithms rely on biometric data — facial characteristics such as hard tissue, curves of the eye socket, nose, and chin.

What began with an identification system of people in photographs on social networks in 2020 is a popular technology that keeps integrating deeper into the lives of millions of people around the world.

How Facial Recognition Works?

The FRT algorithm consists of two stages: identification and verification. Typically, the sequence of the algorithm’s actions is as follows:

1. Face detection. The algorithm highlights the person’s face in the image.
2. Facial features detection. The algorithm calculates the anthropometric points. The system finds features of the face that define individual characteristics. Previously, the main reference point for algorithms was the eyes, but as FRT evolves, at least 68 points on the face are considered nowadays.
3. Face normalization. The FRT system does additional image transformations (head tilt removal, face color correction, etc.) to obtain a clear frontal image.
4. Feature extraction and descriptor computation. A descriptor is calculated — a set of characteristics that describe a face regardless of extraneous factors (age, hairstyle, makeup). Comparing different descriptors makes it possible to assess whether two received facial images refer to the same person.
5. Verification. The resulting digital template is compared with the known faces in the database to complete the identification process.

Video

How Accurate is Facial Recognition?

In December 2019, the National Institute of Standards and Technology (NIST) issued a report on FRT. According to their research, facial recognition algorithms showed better results across demographic groups, spotting “undetectable” characteristics of their representatives.

This study has raised questions regarding the algorithms’ recognition accuracy and the presence of ethnic bias. For instance, one of NIST’s test results demonstrated false match rates of 0.1% for black women and 0.025% for black males, respectively 10 and 2.5 times higher than with white participants.

However, NIST’s report published in January 2020 has refuted the former assumptions about the algorithms’ bias. In fact, the leading FRT systems did not show signs of a significant demographic bias. Research has shown that 17 of the best FRT neural networks have demonstrated very similar recognition accuracy, regardless of skin color and gender. The figures have shown false-negative rates of 0.49% or less for black females and no more than 0.85% for white males.

Besides, NIST’s Face in Video Evaluation (FIVE) study, dated March 2017, has shown that the highest accuracy of FRT algorithms nowadays is achievable with the cooperation of humans and technology.

According to research published in April 2020 by the Center for Strategic and International Studies (CSIS), FRT systems have nearly absolute precision in ideal conditions, reaching a 99.97% recognition accuracy level. However, perfect conditions are hardly achievable in daily operations, and algorithms face various factors affecting their accuracy.

Factors Affecting Face Recognition Accuracy

When choosing a face recognition system, the biggest misconception is that the quality of the system’s work directly depends on the choice of an FRT algorithm. There are several types of recognition algorithms commonly used nowadays:

• Appearance-based. These algorithms do not focus on facial geometry and reflectance.
• Feature-based. Such FRT systems use local face features, such as nose, eyes, and mouth.
• Hybrid. It is a complex method based on the mixture of the two mentioned above, which were created to reduce the false-match rate.

However, even the top neural network is not enough to automatically get error-free results. As proven in Proceedings of ICRIC 2019, high-quality recognition is a complex result that depends on various intrinsic and extrinsic factors.

Aging

Aging is a natural process that has a notable impact on the accuracy of FRT systems. The skin texture changes shift the facial highlights.

As critical changes, such as wrinkles and face shape alteration manifest, a person may become unrecognizable for an FRT algorithm.

Facial Expression

Even a small shift of the facial features can confuse a neural network. When a person smiles, laughs, or cries, the geometry of the face typically changes.

The perception of mimics is a major challenge for an FRT algorithm. Researchers are continuously working on training algorithms in emotion recognition and human identification, regardless of changing facial expressions.

Plastic Surgery

Surgical intervention is a common way of subjectively improving one’s appearance. It may alter any part of the human face.

Although plastic surgery is a widely accepted practice, FRT algorithms have yet to learn to identify people who have gone through such modifications. As of now, there is no neural network capable of recognizing a person after severe reconstructive surgery.

Occlusion

Partial occlusion of the face can happen due to wearing a medical mask, sunglasses, specs, earrings, and scarves. It may also occur due to hair, mustache, or a beard.

Occlusion deteriorates all the existing FRT algorithms. Nevertheless, researchers suggest various methods to help overcome these issues.

Low Resolution

Low-resolution images often derive from surveillance cameras. Individuals caught in such photos often are a crucial part of an investigation but are challenging to identify compared to a high-resolution database.

A low resolution has a substantial negative impact on the recognition rate. However, experts are improving the existing algorithms’ accuracy when dealing with images of low quality.

Noise

Noise is a common characteristic of digital images. The most common types of noise in image processing are Gaussian, Poisson, Speckle, and Salt and Pepper noise.

Pre-handling is compulsory for all the mentioned image noise categories for modern algorithms to perform their task with low error rates.

Illumination

The way the light and shadow fall on a human face always affects the appearance. Thus, photos taken in varying lighting conditions may be potentially confusing for an FRT algorithm.

Even though illumination distortion increases error rates of modern algorithms, there are ways of coping with its effect to receive accurate facial recognition results.

Pose Variation

FRT provides the most accurate results with a frontal face view. Different poses harm face recognition accuracy since most databases nowadays do not store non-frontal individual features.

Pose variation is one of the major issues FRT is facing today. Options of coping with it are being discussed for the sake of the algorithms’ improvement.

How Has Facial Recognition Accuracy Improved?

Most of the research conducted in the FRT sphere in modern days focuses on making it invariable to intrinsic and extrinsic factors. According to the ongoing Face Recognition Vendor Test (FRVT) conducted by NIST, developers aim FRT algorithms to learn invariance by employing large volumes of images.

As of now, all FRT systems require human assistance to compare two images in question to provide a definitive decision on the matter. However, human review is also prone to errors. Nevertheless, FRT systems have substantially gained accuracy since 2013.

According to tests carried out by NIST, a leading algorithm in 2018 makes 20 misses fewer than the best algorithm five years before that did. It is primarily due to the replacement of outdated algorithms by a convolutional neural network (CNN) solution.

The leading FRT algorithms nowadays have almost reached perfection in human identification with an error rate of 0.45% at rank 1. However, most algorithms are still far from achieving such impressive results. Besides, similar to other biometric identification methods, the accuracy rate varies widely throughout different industries.

How is Facial Recognition Accuracy Measured?

FRT’s vector of development is attracting more and more interest from the commercial sector and the state. However, the correct measurement of such a system’s accuracy is challenging and has a lot of nuances.

Almost all modern facial biometrics software is built on machine learning. Face recognition algorithms are trained on large datasets. Both the quality and nature of these datasets have a significant impact on accuracy. The better the initial data, the better the algorithm will cope with the task.

A natural way to test how accurately a face recognition system works is to measure the recognition accuracy on a separate test dataset. Ideally, the dataset should be similar to the images which the system will process in the future. The more similar the test dataset is to the potential data, the more reliable the test results will be. One example of this approach is the NIST Ongoing Face Recognition Vendor Test.

Facial verification can be viewed as a process of making a binary decision: “yes” (two images represent the same person) or “no” (the photographs show different people). Before dealing with verification metrics, it is helpful to understand how we can categorize errors in such tasks. Considering that there are two possible algorithm responses and two options for the actual state of affairs, there are four possible outcomes:

• True positive. The algorithm correctly identified the same person in two images.
• True negative. The FRT system accurately distinguished two different people in the images.
• False positive (type I error). The algorithm identified two different people as the same individual.
• False negative (type II error). The FRT system identified the same person as two different individuals.

Why is a Threshold Important?

Given that there are different types of errors associated with various risks, manufacturers of face recognition software often provide the option to customize the algorithm to minimize one of the error types. For that, the system returns not a binary value but a number that reflects its confidence in the decision. In this case, the user can independently set a threshold and fix the potential error.

As a result, the threshold can be:

• High — Best for businesses with critical uses cases.
• Low — Suitable for companies with non-critical uses cases.

Is Face Recognition More Accurate Than The Human Eye?

Modern researchers’ goal in the field of facial recognition is to create an automated system that can equal or exceed the ability of the human brain to recognize faces.

A NIST research study published in January 2020 showed case studies where human specialists’ capabilities and automated algorithms were compared. Forensic facial examiners and trained facial reviewers went through a test of facial identification. The results of a leading algorithm matched those of the most skillful humans. Nevertheless, an optimal level of FRT accuracy was met only in machine-human collaboration.

Risks Faced with Facial Recognition

The development of FRT is forcing humanity to reconsider everything it knows about confidentiality and the individual’s right to privacy. Besides, FRT is still a developing technology that is far from being perfect. Here are some of the risks associated with facial recognition algorithms.

Identification Errors

Facial recognition systems are not always able to accurately match face prints to the database. As a rule, errors occur due to poor image quality or lack of information in the database.

Confidentiality

Many are concerned about the privacy issues that come with facial recognition. Technology is everywhere, which makes some people uncomfortable. Also, major data breaches are all too common these days, so the personal information that facial recognition software collects is not always ensured.

Data Abuse

While the Pew Research Center found that as of 2019, 56% of Americans trust law enforcement to use facial recognition responsibly, many people are still unsure whether this data will be used ethically.

The study also found that while most consider it acceptable for the authorities to use facial recognition technology to assess security threats in public places, they do not see it as fair to implement it in other areas, such as apartment buildings and workplaces. Such mistrust seems to derive from the fear that private companies may misuse the data.

Summary

Face recognition is a powerful tool that is becoming more and more acceptable globally. FRT demonstrates rapid growth and offers a variety of useful features applicable in all spheres of life.

In recent years, the technology has been raising questions mainly due to its error rates. However, the mere existence of false results does not mean that facial recognition technology has fundamental flaws. Instead, it emphasizes the importance of following the recommended practices when implementing it, such as setting reasonable similarity thresholds to suit a particular case.

Besides, one of the advantages of this technology is its continuous improvement, so the error rates will keep decreasing over time.

Face Recognition Accuracy FAQ