Emotion recognition: introduction to emotion reading technology

What is Emotion Recognition? Meaning & Definition
What Kind of Emotions Can Be Detected and Recognized?
Deep Learning for Emotional Detection & Recognition
How Facial Emotion Recognition Works?
How Is Emotion Recognition Software Trained?
Different Fields for Emotion Recognition
Video
Image
Speech
Text
Conversation
What Is Emotion Recognition Used for Today?
Security Measures
HR Assistance
Customer Service
Differently Abled Children
Audience Engagement
Video Game Testing
Healthcare
What are the Challenges of Emotion Detection & Recognition?
The Moral Problem of Emotion Recognition
Are Emotion Detectors Effective?
Racial Differences in Facial Emotion Recognition
Summary: Is there a Future of Artificial Intelligence (AI) Emotion Recognition
Emotion Recognition FAQ
What is facial emotion recognition?
Can AI detect emotions?
What is emotion recognition training?
Why is emotion recognition important?

What is Emotion Recognition? Meaning & Definition

Emotion recognition is one of the many facial recognition technologies that have developed and grown through the years. Currently, facial emotion recognition software is used to allow a certain program to examine and process the expressions on a human’s face. Using advanced image dispensation, this software functions like a human brain that makes it capable of recognizing emotions too.

It is AI or “Artificial Intelligence” that detects and studies different facial expressions to use them with additional information presented to them. This is useful for a variety of purposes, including investigations and interviews, and allows authorities to detect the emotions of a person with just the use of technology.

What Kind of Emotions Can Be Detected and Recognized?

Emotion recognition can detect and recognize different facial expressions using Facial Expression Analysis. Below is a table showing emotions along with their common corresponding facial expressions:

Deep Learning for Emotional Detection & Recognition

Deep learning is an AI facial recognition function that works like the human brain by processing data and developing patterns used for detecting objects and even in decision making. It is a subset of machine learning and artificial intelligence technology.

Deep learning is based on Neural Net. Neural Net is an algorithm inspired by the structure of the cerebral cortex and functions like the brain. It has become popular over the years and even more lately due to its emotion recognition benefits.

Just like the cerebral cortex, the neural net has several layers: the input layer, the hidden layer, and the output layer. Preferred data can be placed in the neural net, and all of it gets passed through these layers. Each layer modifies all input values and tries to transform them into the target and preferred output.

How Facial Emotion Recognition Works?

Facial emotion detection technology is becoming more and more advanced every year. The AI used detects and studies the expressions depending on many factors to conclude what emotion the person is showing. Factors such as:

• Location of the eyebrows and eyes
• Position of the mouth
• Distinct changes of the facial features

The study held in 2012 regarding emotion recognition summarized the system’s algorithm as follows:

1. Knowledge Base

   This base contains images that are used for comparison and recognizing emotion variations. The images are stored in the database. Every time an input is given to the system, it finds a relevant image from its knowledge base by comparing the stored pictures and the input to come up with an output.

2. Preprocessing and Resize

   This step enhances the input and removes different types of noises. After that, the input image will be resized, typically with the use of the eye selection method.

3. Difference Measurements

   During this step, the system will find any differences between the input image and the stored images and will finally lead to the emotion recognition step.

4. Emotion Recognition

   This is the final step of the process. The comparison is made, and the final output is given depending on the differences found.

How Is Emotion Recognition Software Trained?

The software for emotion detection undergoes training to ensure that outputs are correct and appropriate. Understanding the inputs and outputs is essential for algorithms. Thus, the algorithms must recognize human emotions. There are two approaches used to achieve that:

• Categorical
  Based on this approach, emotions fall into various set classes, and there are a finite set of emotions. Emotions included are contempt, sadness, surprise, disgust, anger, happiness, and fear.
• Dimensional
  Emotions cannot be defined concretely and exist on a spectrum based on this approach. The PAD emotional state has three dimensions, while the Circumplex model of affect uses two.

The choice of approach has some important consequences in modeling them with Machine Learning. When a categorical model of human emotion is chosen, a classifier would likely be created, and texts and images would be labeled with human emotions like “happy”, “sad”, etc. However, when a dimensional model is chosen, outputs would need to be on a sliding scale.

Different Fields for Emotion Recognition

Choosing what model or training data to use is not the end of it, as there are different inputs that can be used or given to the system to analyze. Below are the main fields for emotion recognition:

Video

It is one of the major datasets in Affective Computing that are available, and lots of research is still ongoing to understand how to use them in emotion recognition. An example of modern software for this field is cameras on mobile phones.

Image

Just like video, it is one of the available major sets in emotion recognition. Some research stated that classified emotions in images could be applied in automatic tagging of pictures with emotional categories and sorting video sequences into various genres.

Speech

Usually, speeches are transcribed into texts to be able to analyze them, but this method would not be applicable in emotion recognition. Various research is still ongoing to explore the usage of speech instead of transcribed texts for emotion recognition applications.

Text

Document-Term Matrix or DTM is the usual data structure used for texts. It is a matrix where records of the frequency of words in a document can be found. However, it is not appropriate for determining emotions since it uses individual words. A text is perceived based on its tone, punctuation, etc., and these factors are all being considered by researchers in their ongoing research of new data structures for texts.

Conversation

Emotion recognition in conversation focuses on acquiring emotions from discussions between two or more persons. Datasets under this are commonly from free samples from social platforms. However, there are still a lot of challenges in this field, including the presence of sarcasm in a dialogue, the emotion shift on the interlocutor, and conversational-context modeling.

What Is Emotion Recognition Used for Today?

Nowadays, emotion recognition is used for various purposes that some people do not even notice on a daily basis. Here are some of the areas that would show that emotion recognition is beneficial:

Security Measures

Emotion recognition is already used by schools and other institutions since it can help prevent violence and improves the overall security of a place.

HR Assistance

There are companies that use AI with emotion recognition API capabilities as HR assistants. The system is helpful in determining whether the candidate is honest and truly interested in the position by evaluating intonations, facial expressions, keywords, and creating a report for the human recruiters for final assessment.

Customer Service

There are systems launched nowadays that are installed in customer service centers. Using cameras equipped with artificial intelligence, the customer’s emotions can be compared before and after going inside the center to determine how satisfied they are with the service they’ve received. And if there is a low score, the system can advise the employees to improve the service quality.

Differently Abled Children

There is a project using a system in Google Glass smart glasses that aims to help autistic children interpret the feelings of people around them. When a child interacts with other people, clues about the other person’s emotions are provided using graphics and sound.

Audience Engagement

Companies are also using emotion recognition to determine their business outcomes in terms of the audience’s emotional responses. Apple also released a new feature in their iPhones where an emoji is designed to mimic a person’s facial expressions, called Animoji.

Video Game Testing

Video games are tested to gain feedback from the user to determine if the companies have succeeded in their goals. Using emotion recognition during these testing phases, the emotions a user is experiencing in real-time can be understood, and their feedback can be incorporated in making the final product.

Healthcare

The healthcare industry sure is taking advantage of facial emotion recognition nowadays. They use it to know if a patient needs medicine or for physicians to know whom to prioritize in seeing first.

What are the Challenges of Emotion Detection & Recognition?

Just like any developing technology, emotion recognition is not perfect and has its imperfections and challenges. One of the challenges is that datasets are labeled by the people, and different persons can read and interpret emotions in different ways. Also, some visible visual cues like furrowed eyebrows can mean other emotions aside from anger, and other cues may be subtle hints of anger, although they are not obvious.

Another issue faced by this technology is when detecting emotions from people of different colors. There are models that detect more anger in black people. This means that training sets need to be more diverse, and experts are already doing what they can to fix this.

The Moral Problem of Emotion Recognition

Aside from the challenges it imposes, emotion recognition seems to be also causing moral problems as it sometimes may invade personal space.

There are places where emotion recognition is prohibited by law since people do not like AI interpreting their emotions. There are also places, like California, where the law enforcers are banned from using such technology since it violates the right of the citizens and can lead to them being wary of the authorities.

Are Emotion Detectors Effective?

The emotion recognition technologies are not near to perfect. Although they can truly detect emotions, there are still issues and challenges they face and produce. For example, a system can consider subtle emotions and expressions more alarming than those which actually are. Also, since it automatically links facial expressions to certain emotions, it cannot distinguish which ones are genuine and which are not and could be deceived easily.

AI also cannot easily understand the differences in cultures when expressing emotions and therefore making it hard to produce correct conclusions.

Overall, biases in emotion detection like these, if not addressed, can lead to severely wrong assumptions and cause major misunderstandings. Risks of misinterpretations like these should be considered, and further improvements shall be made.

Racial Differences in Facial Emotion Recognition

Cultural differences can mean varying ways of displaying emotions. These differences pose an issue when using facial emotion recognition on people of different races. Display rules in Southeast Asia might be different from those in the UK.

Lisa Feldman Barrett, a psychology professor at Northeastern University, admits that it doesn’t make sense to map facial expressions directly onto emotions across all cultures and contexts. She said that a person might scowl when they’re angry, but another one might smile politely while planning on how to make their enemy fall. That is why having diverse data sets is necessary since, in different countries, emotional expression seems to take on different intensities and nuances.

Summary: Is there a Future of Artificial Intelligence (AI) Emotion Recognition

A study published in 2015 showed that recognition results improve when confounding factors are removed from the input images and the lighting is adequate.

AI Emotion Recognition technology is continuously being studied and improved to solve and provide solutions to arising risks and issues and to avoid any cultural and ethical problems in the society.

Emotion Recognition FAQ

What is facial emotion recognition?

Facial emotion recognition is an AI technology used to analyze a person’s face and interpret their expressions. Using these analyses and interpretations, the emotion a person is experiencing can be concluded with the use of technology. Emotion recognition can be done using texts, images, videos, speeches, and conversations.

Can AI detect emotions?

With the emotion recognition system, AI can detect the emotions of a person through their facial expressions. Detected emotions can fall into any of the six main data of emotions: happiness, sadness, fear, surprise, disgust, and anger. For example, a smile on a person can be easily identified by the AI as happiness.

What is emotion recognition training?

To conclude what emotion the detected expressions correspond to, training must be done. It is the process of using data sets in recognizing the emotion detected by the system. The approach is usually categorical but can also be dimensional.

Why is emotion recognition important?

Emotion recognition provides benefits to many institutions and aspects of life. It is useful and important for security and healthcare purposes. Also, it is crucial for easy and simple detection of human feelings at a specific moment without actually asking them.