How Facial Recognition Works: Technology Explained in Detail

What Is Facial Recognition?
Brief History of Facial Recognition
Examples of Face Recognition Systems & Technologies
How Does Facial Recognition Work?
Step 1: Face Detection
Step 2: Face Analysis
Step 3: Converting an Image into Data
Step 4: Matching
Face Recognition Algorithms
Where Face Recognition Is Used
Facial Recognition Threats & Concerns
Privacy
Misuse of Data
Errors in Identification
How to Protect Yourself Against Face Recognition
Facial Recognition Limitations
The Future of Face Recognition
Facial Recognition Solution from RecFaces
Face Recognition FAQ
How accurate is facial recognition?
Is facial recognition legal?
Why is facial recognition controversial?
How does face recognition work when you wear a mask?
Recommended reading

What Is Facial Recognition?

Facial recognition is a software algorithm used to verify or identify an individual’s identity by processing a video frame or a digital image in which the human’s face is visible. There are several different methods that facial recognition technologies work, but they generally compare facial features in an image to faces contained within a database.

Facial recognition has many practical uses. For instance, police officers can use such technology to identify people that they pull over. Previously, facial recognition software could only be used as a computer application. Nowadays, though, it can be used on mobile devices and on other kinds of technology, like robotics. This opens up many new ways in which facial recognition can be applied. It’s recently become quite popular as a marketing and commercial identification tool. Some other applications include automatic image indexing, video surveillance, human-computer interactions, and more.

Brief History of Facial Recognition

Facial recognition was first pioneered in 1964 by Woody Bledsoe, along with Charles Bisson and Helen Chan Wolf. Because their project was funded by an anonymous intelligence agency, the majority of the trio’s research and work was never made public. However, it was later disclosed that their early work included manual mapping of facial features, and then mathematically rotating them with a computer.

In the 1970s, the accuracy of facial recognition was further evolved to account for lip thickness, hair color, and 19 other specific identifying markets. Come the 1980s and 1990s, linear algebra began to be applied to facial recognition technology. A decade later, the National Institute of Standards and Technology and the Defence Advanced Research Projects Agency coordinated to create a facial image database.

In 2006, the Face Recognition Grand Challenge was launched — it was designed in order to support the U.S. government’s efforts to use facial recognition. The FRGC studied and rated modern facial recognition algorithms and found that they had become 100 times more accurate over the last decade.

From 2010 to the present time, we began to run into face recognition more and more in our daily lives. For instance, there is Facebook’s algorithm for tagging faces, the iPhone’s new capabilities to unlock via FaceID, and face scanning at concerts, stadiums, airports, and more.

Examples of Face Recognition Systems & Technologies

These are three well-known companies in the facial recognition industry:

FaceNet — This artificial neural network was published in 2015 by Google researchers. It has an extremely high accuracy rate of 99.63%. It’s used in Google Photos to automatically tag photos in which a person’s face is recognized. FaceNet uses the “Labelled Faces in the Wild” dataset which is a public benchmark for facial verification.

FaceApp — In mid-2019, the FaceApp craze swept the globe. It is an app that implements facial recognition for pure entertainment purposes. Users could take a selfie and change their facial features, seeing how they would look older, younger, or even as the opposite gender. It could realistically change your hair color, apply lipstick, add a beard, and change other specific details about your appearance.

Face ID — This is a facial recognition technology that was designed by Apple and released in 2017. This technology was created for security purposes and has been introduced to most new iPhone models, as well as all iPad Pro models. Users can unlock compatible devices by using the front-facing camera. Face ID uses 3D modeling of the user’s face, so it can’t be tricked by wearing a mask or showing a photo of the phone’s owner. The faceprint maps are quite in-depth; over 30,000 variables are captured and compared. Face ID can be used as an authentication token throughout the entire Apple ecosystem, allowing you to authorize purchases in the App Store, the iTunes Store, the iBooks Store, and Apple Pay.

How Does Facial Recognition Work?

Any facial recognition algorithm uses biometrics to map out facial features captured in a video still or a photograph. That information is then compared to a database of faces. There are four general steps in the process, which we’ll explain further.

Step 1: Face Detection

First, a camera will detect and recognize a human’s face — one that can either be in a crowd or alone. It is most easily detected when the person is looking straight at the camera. However, modern technological advances allow face recognition software to still work if the person’s face is angled slightly.

Step 2: Face Analysis

After detection and recognition, a photo will capture the face and will then be analyzed. The majority of face recognition technologies use 2D images instead of 3D. This is because 2D photos are more readily correlated with public photos or pictures in a database (these are typically 2D as well). During analysis, the face will be separated into distinguishable landmarks — we can call these nodal points. A human face has eight nodal points. Face recognition technology will analyze each of these points — for example, the distance between your eyebrows.

Step 3: Converting an Image into Data

After analysis, each nodal point becomes a number in the application database. The entire numerical code is referred to as a faceprint. Just like how everybody has a unique thumbprint, everyone also has a unique faceprint.

Step 4: Matching

The final step of the process is finding a match. Your faceprint is compared to a database of other facial codes. The number of faces that are compared depends on the database and how many databases the software has access to. For instance, the FBA has access to 21 state databases, with 641 million photos across them. The facial recognition technology then identifies a match for your exact facial features — it returns the user with the found match and other relevant information, such as an address and a name.
Watch the video

Face Recognition Algorithms

There have been a number of methods developed to make facial identification easier and more accurate. One of these is gradients — essentially, this replaces the image taken with a version that emphasizes the details most relevant to face recognition. In this case, every pixel is replaced with a relative lighting representation, meaning that it shows how the pixel’s brightness compares to the surrounding pixels. This relative measurement makes it easier to identify the same face throughout multiple forms of lighting.

Another approach is projection. This is when a 2D photo is projected onto a 3D model — say, for instance, a cylinder. When a face is wrapped around a 3D model, distinguishing characteristics are revealed that would be more difficult to find in a static and flat image. Projection lets you overcome the difficulties inherent to 2D facial recognition. With this technology, you don’t need an image taken under ideal conditions in order to achieve great precision.

Where Face Recognition Is Used

Facial recognition can be applied in many areas, some of which have already been mentioned. It doesn’t just have to be used for security purposes, although that is the main reason. Some applications include:

• Mobile device owner authentication
• School security
• Airline customs
• Entertaining apps
• Forensic science
• Recognizing shoplifters

Facial Recognition Threats & Concerns

Even though facial recognition has helped many industries to step up their security, technology raises large concerns for many people. This, coupled with the fact that the surveillance market is growing exponentially and that face recognition systems can be found in almost every aspect of life, there is a significant amount of people who are against this kind of technology. Let’s take a look at some of the main concerns regarding face recognition software.

Privacy

Facial recognition technology is gaining momentum — but, at the same time, rigorous new privacy laws are being enacted. For instance, the EU’s General Data Protection Regulation restricts newer commercial projects. Anita Bapat, a privacy partner at the law firm Kemp Little, has said that capturing somebody’s facial features is akin to gathering personal data — that is, if you’re able to use the features for identification features. Thus, organizations that would like to use facial recognition tech must be granted explicit consent from the individual. And, even then, the consent can be withdrawn.

Misuse of Data

There is much concern that companies will abuse the data that they collect via facial recognition. And, even if a company has the best intentions, it is always possible that the data will be misused if a data breach occurs.

Errors in Identification

Face recognition technology is not accurate 100% of the time. There is a higher bias when women and people of color are identified, especially when an outdated algorithm is used. As facial recognition is often used by law enforcement, errors in identification could lead to false incrimination.

How to Protect Yourself Against Face Recognition

There are several ways that you can mask your appearance in order to prevent facial recognition software from identifying you. Some of these methods include:

• Strategically applying makeup and styling your hair. By doing these actions in a certain way, you can actually undermine a face recognition system’s ability to analyze your facial features. The first main reason is because you can block substantial portions of your face. The system might not even recognize both scanning you if they don’t realize that it is a face. Facial recognition systems depend partially on analyzing the way that light hits your face. For instance, your cheekbones are nearly always more lit up than your eye sockets. But with makeup, you can change what features of your face are highlighted and which ones are shadowed. You can also use makeup and hair styles to cover up moles, scars, and other marks that can be used to identify you.
• Blocking the shape of your lips. Facial recognition technology tends to analyze people’s lips more than other features. Growing a mustache long enough to cover your upper lip can reduce the chance of being identified. If you can’t grow a mustache, try using a little bit of concealer to strategically change the shape of your lips or wearing a face mask.
• Wearing a brimmed hat. If you walk with your head down while wearing a baseball cap, your face will be totally blocked from ceiling-mounted cameras. We also recommend tucking the tops of your ears into the side of your cap.
• Blocking one or both eyes. Face recognition systems analyze the positions, shape, size, color, and distance between your eyes. By wearing reflective sunglasses, you can interfere with the system’s ability to see the area surrounding your eyes.

These tips all apply for when you are out and about. But what about social media face recognition technology? If you have a Facebook account, go into your account settings, and opt-out of their facial recognition system. Google+ also has a similar system that can be turned on and off; unlike Facebook, though, you have to opt-in for it to apply to your account.

Facial Recognition Limitations

In just a few years, the capabilities and accuracy of facial recognition technologies have increased dramatically. However, there are still limitations — especially when people implement the tips listed above. In ideal conditions, a facial recognition system can have 99.97% accuracy. However, in the real world, ideal photos are rarely achieved. Lighting and positioning have to allow the subject’s facial features to be unobscured and clear.

Aging can also increase error rates. As a subject’s face ages over time, the system will be less likely to match it with their photo from a database.
Another issue is the wide variation in accuracy between vendors. Some vendors have developed algorithms that deliver highly accurate results. However, the average market provider is not so accurate. Face recognition software is still a long way off from being commoditized.

The Future of Face Recognition

The future of facial recognition is incredibly promising. Even though there are still problems with consistent levels of accuracy, we are still leaps and bounds ahead of where we were 5-10 years ago. This industry is going to continue growing and it will pave the way for massive revenue potential in the next several years. The major industries that will be intensely influenced by this technology include surveillance and security. However, schools, universities, and hospitals will be close behind.

Some countries are further ahead of the curve than others in regard to implemented technology. For instance, face recognition technology is used in India in order to prevent ATM fraud. The U.S.has applied facial recognition to airport security in order to identify and register visitors. There are also several states within the US that enable law enforcement to run face recognition searches, matching people’s photos against IDs and driver’s licenses.

Facial Recognition Solution from RecFaces

Are you ready to get ahead of the curve and begin implementing facial recognition technology in your workplace? RecFaces has the perfect solution for you. We develop biometric face identification systems that can solve real business problems, increase the effectiveness of your security team, provide higher quality customer service, and automate processes. Some of our top products include:

• Id-Guard: This is a biometric identification plugin developed for leading CCTV, and access control systems. It is designed to help security teams increase the speed in which they respond to incidents and conduct investigations.
• Id-Gate: This is a biometric software developed to enrich the functions of existing or new access control systems with additional modern functions using facial recognition technology. Designed to verify access rights of a company’s employees and visitors, Id-Gate uses biometric face identification instead of unreliable key- and RFID-cards. 
• Id-Welcome: With this software product, you can use biometric identification to give personalized welcome messages to subjects, raising the welcoming atmosphere of your business.
  Besides these, we have many other products available at recfaces.com. All of our products are highly beneficial when used in banks, retail, business centers, transport facilities, educational institutions, and many more enterprises.

Face Recognition FAQ

We hope that this comprehensive guide on facial recognition has been helpful. To assist further, we’ve compiled a list of questions that we are frequently asked by readers.

How accurate is facial recognition?

Under ideal circumstances, face recognition can be 99.97% accurate. However, with poor lighting and positioning, leading algorithms can have a 9% mismatch rate — and less effective algorithms will have higher error rates.

Is facial recognition legal?

This depends on your country of residence. A growing number of countries are employing facial recognition technology in a fully legal manner — including the U.S., China, Germany, Ukraine, and many more.

Why is facial recognition controversial?

Some people are concerned that facial identification is a breach of privacy. Furthermore, it is possible that the databases used by face identification systems could be hacked — and the data would thus be misused. Finally, a significant number of people don’t have a specific reason for disliking facial recognition, except for just having a gut feeling.

How does face recognition work when you wear a mask?

Throughout the summer of 2020, face masks were “breaking” facial recognition systems — after all, they blocked out a large portion of the subject’s face. However, developers are reworking algorithms to account for masks. Not all facial recognition systems can successfully identify somebody who is wearing a mask, but some systems have adapted and can do so with a decent amount of accuracy.

Recommended reading

• What Is Facial Recognition Used For