There are two primary ways of communication between human beings; verbal and non-verbal. Facial expressions are critical in non-verbal communication since humans can recognize and comprehend expressions. Since the use of smart technology is increasing, the facial emotion recognition system is becoming one of the vital components in human-machine interaction. In a basic form of the facial expression recognition system, input sensing devices are used. These devices can be a webcam that takes a picture of the subject. The face is detected, and the representative features are extracted. After that, pre-processing is carried out, and the expression is grouped into emotions such as happy, neutral, sad, or hungry using the classifier. This paper discusses psychological sensors used to detect facial features and movements to extract the psychological state of an individual.
According to Feidakis et al. (2011), sixty-six emotions are categorized into two primary groups; ten basic emotions and the fifty-six secondary ones. Evaluating these emotions is complex, primarily when automated recognition and evaluation methods are required. Most challenges in their evaluation are experienced due to their overlapping parameters. To overcome this, psychological sensors evaluate the expressions based on the dimensions of the emotions, either high or low (arousal) and negative or positive (activation). The expressions are also analyzed by considering only the primary emotions, which can be easily defined. Most researchers and sensors used Russel’s circumplex model of emotions, which distributes the primary ten emotions in two-dimensional; arousal and valence (Russell,1980, p. 1162).
Figure 1:Russel’s circumplex model of emotions
Sensors and methods
Automated emotion recognition analyses the above-stated emotions by determining electric impulses from the nervous system or parameters of the human body. The common methods used are electroencephalography, eye activity, blood pressure, skin resistance measurements, and motion analysis.
Electroencephalography (EEG)
It is used to determine electrical impulses from the human brain. This technique was introduced by a German psychiatrist, Hans Berger, in 1924 (Britton et al., 2016). Electroencephalography signals are collected using an electroencephalogram. The primary parts of the device are needle or metal plate electrodes placed directly on the scalp (Dzedzickis et al., 2020, p. 3). The signal is the voltage fluctuation between the paired electrode with respect to time. The amplitude of the signal is evaluated using peak to peak technique.
The brain is exposed to various stimuli to evaluate the psychological state based on facial emotions or expressions. The generated signals are analyzed based on five frequency ranges: gamma, beta, alpha, theta, and delta (Dzedzickis et al., 2020, p. 4). The frequency of gamma (ƴ) is 32Hz, and above, and the related emotional and psychological state is processing and ideation, memory and language, and regional learning. Beta (β) has a 16-32 Hz frequency and exhibits concentration and beware. Alpha (α) has 8-16 Hz and demonstrates relaxation and creativity. Theta (θ) has4-8 Hz and portrays meditation and deep relation. Delta (δ) has 0.5-4 Hz and shows intuition and a strong sense of empathy (Dzedzickis et al., 2020, p. 4).
Electromyogram
This sensory technique is used to record and evaluate the electrical potential created by muscle cells (Dzedzickis et al., 2020, p. 19). In facial recognition system, it is used to assess the relationship between physiological reactions and cognitive emotions. The technique mainly focuses on facial expressions because facial mimicry is hypothesized to contribute to an emotional reaction to different stimuli. The hypothesis was introduced by Friesen and Ekman in 1978 when different correlations between facial expression and emotions were recorded, as shown below (Dzedzickis et al., 2020, p. 19; Shakshi, 2016, p. 1209).
| Actions | Involved Muscles | Emotion |
| Raising eyebrows, depressing lip corners, lowering eyebrows | Frontalis, Corrugator supercilia, Depressor angulioris | Sadness |
| Lowering eyebrows, closing eyelids, raising the eyelid | Orbicularis oculi, Levator palpebrae superioris, | Anger |
| Raising eyebrows and eyelids, lowering eyebrows. | Frontalis, Levator palpebrae superioris, Corrugator supercilii, | Anger |
| Twisting corners of the mouth, closing eyelids | Orbicularis oculi, Zygomaticus major | Happiness |
| Raising upper lip and wrinkling nasal skin. | Levator labii superioris alaeque nasi. | Disgust |
The electromyogram process is done by determining the voltages between electrodes. The process consists of two primary steps: defining the baseline by measuring the voltage when a human is calm and then measuring the reaction to stimuli and evaluating the caused effect (the ratio between measured value and baseline) (Dzedzickis et al., 2020, p. 4).
Conclusion
Facial expression recognition is a valuable and powerful technique for analyzing an individual’s emotional and psychological state. The correlation between psychological state and human body reactions such as facial expressions has been known for a long. However, there are many uncertainties in identifying the suitable sensor and analysis technique. For Facial expressions, electromyogram and Electroencephalography have proven to be the most effective.
References
Britton, J.W., Frey, L.C., Hopp, J.L., Korb, P., Koubeissi, M.Z., Lievens, W.E., Pestana-Knight, E.M. and St Louis, E.K., 2016. Electroencephalography (EEG): an introductory text and atlas of normal and abnormal findings in adults, children, and infants. https://europepmc.org/article/nbk/nbk390354
Dzedzickis, A., Kaklauskas, A. and Bucinskas, V., 2020. Human emotion recognition: Review of sensors and methods. Sensors, 20(3), p.592. https://www.mdpi.com/1424-8220/20/3/592
Russell, J.A. A circumplex model of effect. J. Pers. Soc. Psychol. 1980, 39, 1161–1178.
Shakshi, R.J., 2016. Brain wave classification and feature extraction of EEG signal by using FFT on lab view. Int. Res. J. Eng. Technol, 3, pp.1208-1212.
write