Using Digital Image Speckle Correlation (DISC) to Identify Micro Emotions

Facial expressions are one of the most basic and natural forms of communication between humans. Alongside general facial expressions, micro-emotions are spontaneous changes in facial muscles that indicate hidden emotions. Micro-emotions become apparent in response to specific emotion-inducing stimuli, such as pictures and videos. These fleeting expressions are low in intensity and are, therefore, significantly more difficult to identify than visible facial expressions and emotions.¹ This study uses Digital Image Speckle Correlation (DISC) to identify micro-emotions by analyzing subcutaneous facial muscle displacement through a contact-free, non-invasive method.<br/><br/>To verify the validity of DISC for tracking facial muscle movement, a video was taken of identical twins and a control performing the same facial actions of frowning and smiling. The control was a subject with no relation to the twins. While the twins used similar parts of muscles to perform actions, the control used quite different parts of facial muscles to perform the same actions. For instance, when frowning, the identical twin’s forehead muscles toward the inside of the face. However, the control’s forehead muscles moved downward toward the outside of the face.<br/><br/>Sets of ten images were obtained from the International Affective Picture System (IAPS) and categorized using a valence and arousal scale. Valence denotes the emotional value, while arousal characterizes its level of stimulation. The analysis of the participants' faces shows that high-valence images tend to evoke positive emotions, translating to more pronounced movement in the cheeks and mouth region. When the subject saw an image with positive valence and high arousal, large levels of displacement occurred in those areas. The magnitude of facial displacement in four quadrants of the face was represented through a graph. In the graph, visible spikes span approximately a second. These peaks represent fleeting muscle movements that correlate to micro-emotions, which can be detected through displacement. On the other hand, the negative valence images induced significant amounts of micro-emotions in the upper quadrants, specifically in the forehead and eyebrow areas, commonly associated with reactions to sad emotions.<br/><br/>DISC has the potential to be used to analyze the efficacy of treatments for certain psychological disorders. Major Depressive Disorder (MDD) presents with non-verbal indicators such as the lack of or abundance of specific facial movements and reactions. Hence, changes in facial expressions and movements as identified by DISC can be used to diagnose and understand depression and track the progression and regression of depressive symptoms.² For example, esketamine is approved by the FDA to treat MDD that cannot be treated through other antidepressants.<br/><br/>In particular, a case study involving an esketamine patient before and after receiving treatment was utilized to analyze the effects of this treatment on patients. Analysis of the video images indicated differences in the intensity of the patient's reactions before and within an hour after treatment. The time to react to the different images was also measured and could be a measure of the impact on the information processing ability of the patient. These results will be correlated to the known physiological impact of esketamine as a sedative agent³ and the answers to the self-report questionnaire administered to the patient, such as BDI or HAM-D which are used to assess the severity of depressive symptoms.<br/><br/>¹X. Li et al. "Towards Reading Hidden Emotions: A Comparative Study of Spontaneous Micro-Expression Spotting and Recognition Methods," <i>IEEE Transactions on Affective Computing,</i> vol. 9, no. 4, pp. 563-577, 2018<br/>²Stratou, G. et al. “Automatic nonverbal behavior indicators of depression and PTSD: the effect of gender,” <i>Journal on Multimodal User Interfaces, </i>2015<br/>³Ekman, P. et al. “Facial Expression In Affective Disorders,”<i> What the Face Reveals,</i> 2013