There is debate within the literature as to whether emotion dysregulation (ED) in Attention-Deficit Hyperactivity Disorder (ADHD) reflects deviant attentional mechanisms or atypical perceptual emotion processing. Previous reviews have reliably examined the nature of facial, but not vocal, emotion recognition accuracy in ADHD. The present meta-analysis quantified vocal emotion recognition (VER) accuracy scores in ADHD and controls using robust variance estimation, gathered from 21 published and unpublished papers. Additional moderator analyses were carried out to (...) determine whether the nature of VER accuracy in ADHD varied depending on emotion type. Findings revealed a medium effect size for the presence of VER deficits in ADHD, and moderator analyses showed VER accuracy in ADHD did not differ due to emotion type. These results support the theories which implicate the role of attentional mechanisms in driving VER deficits in ADHD. However, there is insufficient data within the behavioural VER literature to support the presence of emotion processing atypicalities in ADHD. Future neuro-imaging research could explore the interaction between attention and emotion processing in ADHD, taking into consideration ADHD subtypes and comorbidities. (shrink)

This chapter argues that emotion recognition is a skill. A skill perspective on emotion recognition draws attention to underappreciated features of this cornerstone of social cognition. Skills have a number of characteristic features. For example, they are improvable, practical, and flexible. Emotion recognition has these features as well. Leading theories of emotion recognition often draw inadequate attention to these features. The chapter advances a theory of emotion recognition that is better (...) suited to this purpose. It proposes that emotion recognition involves scripts. Emotion scripts describe how people are likely to behave in different emotional contexts. Scripts can be improved with the addition of richer social knowledge, they are practical in that they describe and guide social interaction, and they are flexible because they must accommodate the fact that emotions have different behavioral impact depending on the agents involved and the circumstances at play. Learning to recognize emotions through scripts qualifies as a skill. (shrink)

According to a dominant interpretation of the simulation hypothesis, in recognizing an emotion we use the same neural processes used in experiencing that emotion. This paper argues that the view is fundamentally misguided. I will examine the simulational arguments for the three basic emotions of fear, disgust, and anger and argue that the simulational account relies strongly on a narrow sense of emotion processing which hardly squares with evidence on how, in fact, emotion recognition is (...) processed. I contend that the current body of empirical evidence suggests that emotion recognition is processed in an integrative system involving multiple cross-regional interactions in the brain, a view which squares with understanding emotion recognition as an information-rich, rather than simulational, process. In the final section, I discuss possible objections. (shrink)

We develop a version of a direct perception account of emotion recognition on the basis of a metaphysical claim that emotions are individuated as patterns of characteristic features. On our account, emotion recognition relies on the same type of pattern recognition as is described for object recognition. The analogy allows us to distinguish two forms of directly perceiving emotions, namely perceiving an emotion in the absence of any top-down processes, and perceiving an (...) class='Hi'>emotion in a way that significantly involves some top-down processes ; and, in addition, an inference-based evaluation of an emotion. Our model clarifies the epistemology of emotion recognition. (shrink)

BackgroundWe review original papers on ovarian-hormone status in two areas of emotional processing: facial emotion recognition and emotional memory. Ovarian-hormone status is operationalized by the levels of the steroid sex hormones 17β-estradiol and progesterone, fluctuating over the natural menstrual cycle and suppressed under oral contraceptive use. We extend previous reviews addressing single areas of emotional processing. Moreover, we systematically examine the role of stimulus features such as emotion type or stimulus valence and aim at elucidating factors that (...) reconcile the inconsistent results.MethodsWe followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines and included papers published until September 2020 indexed in PubMed and Web of Science databases. Search terms were MeSH terms AND AND with representing our separately searched areas, resulting in, and. To be included, articles had to be written and published in English, examine healthy, non-pregnant adult women in their reproductive age, and measure or at least estimate levels of E2 and P4. In PubMed, the search was limited to humans and to the search term present in the title or abstract.ResultsFeatures of the provided stimulus material constitute a relevant influence that interacts with E2- and P4-related ovarian-hormone status. For instance, recognition of basic emotions appears to be more related to P4- than E2-levels. Quite consistent, OC intake was accompanied by impaired recognition accuracy of basic and also complex emotions, although not in a recent large-sample study assessing complex emotions. Memory recall of negative content was mainly enhanced by P4, especially after having been stressed.Discussion and ConclusionWe document the methodological diversity in the field, presumably contributing to the heterogeneity of results. More studies explicitly contrasting the early follicular phase, mid-cycle phase, mid-luteal, and OC intake while standardizing tasks are needed. Research would take advantage of using within-subject designs and accounting for the recognition of complex emotions. (shrink)

Emotion recognition skills and the ability to understand the mental states of others are crucial for normal social functioning. Conversely, delays and impairments in these processes can have a profound impact on capability to engage in, maintain, and effectively regulate social interactions. Therefore, this study aimed to compare the performance of 42 autistic children, 45 unaffected siblings, and 41 typically developing controls on the Affect Recognition and Theory of Mind subtests of the Developmental Neuropsychological Assessment Battery. There (...) were no significant differences between siblings and TD controls. Autistic children showed significantly poorer performance on AR when compared to TD controls and on TOM when compared to both TD controls and unaffected siblings. An additional comparison of ASD, unaffected sibling and TD control subsamples, matched on full-scale IQ, revealed no group differences for either AR or TOM. AR and TOM processes have received less research attention in siblings of autistic children and remain less well characterized. Therefore, despite limitations, findings reported here contribute to our growing understanding of AR and TOM abilities in siblings of autistic children and highlight important future research directions. (shrink)

Linguistic communication is an important part of the cross-cultural perspective, and linguistic textual emotion recognition is a key massage in interpersonal communication. Spanish is the second largest language system in the world. The purpose of this paper is to identify the emotional features in Spanish texts. The improved BiLSTM framework is proposed. We select three widely used Spanish dictionaries as the datasets for our experiments, and then we finally obtain text sentiment classification results through text preprocessing, text (...) class='Hi'>emotion feature extraction, text topic detection, and emotion classification. We inserted the attention mechanism in the improved BiLSTM framework. It enables the shared feature encoder to obtain weighted representation results in the extraction of emotion features, which enhances the generalization ability of the model for text emotion feature recognition. Experimental results demonstrate that our approach performs better for specialized Spanish dictionary datasets. In terms of emotion recognition accuracy, the average value is as high as 76.21%. The overall performance outperforms current comparable machine learning methods and convolutional neural network methods. (shrink)

The purpose of this study was to investigate emotion recognition and processing in patients with mild cognitive impairment in order to update the state of current literature on this important but undervalued topic. We identified 15 papers published between 2012 and 2022 that meet the inclusion criteria. Paper search, selection, and extraction followed the PRISMA guidelines. We used a narrative synthesis approach in order to report a summary of the main findings taken from all papers. The results collected (...) are still ambiguous: some studies did not find any differences between MCI and healthy controls groups in emotion recognition and processing, and other results reported emotion-specific deficits in emotion recognition regarding MCI patients. It is essential to underline that these findings could not be generalized to the whole MCI population due to the heterogeneous use of measures and composition of the sample. This does not allow us to make a comprehensive comparison between the results. Our suggestion for future research is to align the results using the same type of tests and emotion recognition assessment. (shrink)

To define the determinants of impaired facial emotion recognition (FER) in patients with mesial temporal lobe epilepsy associated with hippocampal sclerosis (MTLE-HS), we examined 76 patients with unilateral MTLE-HS, 36 prior to antero-mesial temporal lobectomy (AMTL) and 40 after AMTL, and 28 healthy control subjects with a FER test consisting of 60 items (20 each for anger, fear, and happiness). Mean percentages of the accurate responses were calculated for different subgroups: right vs. left MTLE-HS, early (age at onset (...) <6 years) vs. late-onset, and before vs. after AMTL. After controlling for years of education, duration of epilepsy and number of antiepileptic drugs (AEDs) taken, on multivariate analysis, fear recognition was profoundly impaired in early-onset right MTLE-HS patients compared to other MTLE patients and control subjects. Happiness recognition was significantly better in post-AMTL MTLE-HS patients compared to pre-AMTL patients while anger and fear recognition did not differ. We conclude that patients with right MTLE-HS with age at seizure onset <6 years are maximally predisposed to impaired fear recognition. In them, right AMTL does not further worsen FER abilities. Longitudinal studies comparing FER in the same patients before and after AMTL will be required to refine and confirm our cross-sectional observations. (shrink)

At present, there are so many learners in online classroom that teachers cannot master the learning situation of each student comprehensively and in real time. Therefore, this paper first constructs a multimodal emotion recognition model based on CNN-BiGRU. Through the feature extraction of video and voice information, combined with temporal attention mechanism, the attention distribution of each modal information at different times is calculated in real time. In addition, based on the recognition of learners’ emotions, a prediction (...) model of learners’ achievement based on emotional state assessment is proposed. C4.5 algorithm is used to predict students’ academic achievement in the multi-polarized emotional state, and the relationship between confusion and academic achievement is further explored. The experimental results show that the proposed multi-scale self-attention layer and multi-modal fusion layer can improve the achievement of ER task; moreover, there is a strong correlation between students’ confusion and foreign language achievement. Finally, the model can accurately and continuously observe students’ learning emotion and state, which provides a new idea for the reform of education modernization. (shrink)

Most research on cross-cultural emotion recognition has focused on facial expressions. To integrate the body of evidence on vocal expression, we present a meta-analysis of 37 cross-cultural studies...

Background: Clinical and experimental findings have suggested that insomnia is associated with altered emotion processing, such as facial emotion recognition and impairments in executive functions. However, the results still appear non-consensual and have recently been presented by a few number of studies. Accordingly, the aim of the present study was to investigate whether patients with Insomnia disorder will present alterations in recognition of facial emotions and that such alterations will be related to Executive Functions and that (...) Insomnia Disorder patients will exhibit a lower Executive Functions performance when compared to normal controls individuals. Method: A total of 11 individuals diagnosed with insomnia disorder and 15 control participants took part in the present cross-sectional study. They completed questionnaires covering socio-demographic data, a sleep and mood evaluation, and a cognitive assessment. Moreover, they underwent a computerized Facial Emotional Recognition test, consisting of facial emotion labeling and facial emotion matching. Results: Individuals with insomnia presented a lower recognition of facial expressions of fear and sadness in comparison to control volunteers. Perceptual Organization Index was found to be statistically significant lower in the insomnia group compared to the control group. There was a statistically significant association between facial recognition emotional and cognitive processes of in executive functions performance, especially in processes related to the inhibitory control, planning capacity, problem solving and cognitive flexibility in insomnia group. Conclusions: Insomnia disorder impairs the facial emotion recognition of the fear and sadness. Insomniacs with impairment in the facial emotion recognition had impaired performance of executive functions. There is a negative impact on processes related to cognitive-emotional capacity than on processes related to social-emotional recognition. Keywords: insomnia, facial emotional recognition, sleep deprivation, cognition, executive functions. (shrink)

BackgroundParanoia is associated with a multitude of social cognitive deficits, observed in both clinical and subclinical populations. Empathy is significantly and broadly impaired in schizophrenia, yet its relationship with subclinical paranoia is poorly understood. Furthermore, deficits in emotion recognition – a very early component of empathic processing – are present in both clinical and subclinical paranoia. Deficits in emotion recognition may therefore underlie relationships between paranoia and empathic processing. The current investigation aims to add to the (...) literature on social cognition and paranoia by: (1) characterizing the relationship between paranoia and empathy, and (2) testing whether there is an indirect effect of emotion recognition on the relationship between empathy and paranoia.MethodsParanoia, empathy, and emotion recognition were assessed in a non-clinical sample of adults (n= 226) from the Nathan Kline Institute-Rockland (NKI-Rockland) dataset. Paranoia was measured using the Peters Delusions Inventory-21 (PDI-21). Empathy was measured using the Interpersonal Reactivity Index (IRI), a self-report instrument designed to assess empathy using four subscales: Personal Distress, Empathic Concern, Perspective Taking, and Fantasy. Emotion recognition was assessed using the Penn Emotion Recognition Test (ER-40). Structural equation modeling (SEM) was used to estimate relationships between paranoia, the four measures of empathy and emotion recognition.ResultsParanoia was associated with the Fantasy subscale of the IRI, such that higher Fantasy was associated with more severe paranoia (p< 0.001). No other empathy subscales were associated with paranoia. Fantasy was also associated with the emotion recognition of fear, such that higher Fantasy was correlated with better recognition of fear (p= 0.008). Paranoia and emotion recognition were not significantly associated. The Empathic Concern subscale was negatively associated with emotion recognition, with higher empathic concern related to worse overall emotion recognition (p= 0.002). All indirect paths through emotion recognition were non-significant.DiscussionThese results suggest that imaginative perspective-taking contributes to paranoia in the general population. These data do not, however, point to robust global relationships between empathy and paranoia or to emotion recognition as an underlying mechanism. Deficits in empathy and emotion recognition observed in schizophrenia may be associated with the broader pathology of schizophrenia, and therefore not detectable with subclinical populations. (shrink)

Results from studies on gender differences in emotion recognition vary, depending on the types of emotion and the sensory modalities used for stimulus presentation. This makes comparability between different studies problematic. This study investigated emotion recognition of healthy participants (N = 84; 40 males; ages 20 to 70 years), using dynamic stimuli, displayed by two genders in three different sensory modalities (auditory, visual, audio-visual) and five emotional categories. The participants were asked to categorise the stimuli (...) on the basis of their nonverbal emotional content (happy, alluring, neutral, angry, and disgusted). Hit rates and category selection biases were analysed. Women were found to be more accurate in recognition of emotional prosody. This effect was partially mediated by hearing loss for the frequency of 8,000 Hz. Moreover, there was a gender-specific selection bias for alluring stimuli: Men, as compared to women, chose “alluring” more often when a stimulus was presented by a woman as compared to a man. (shrink)

Purpose This study responds to Agnieszka Landowska’s paper about the lack of accuracy in emotion recognition. Design/methodology/approach The approach is purely theoretical. The paper also refers to empirical studies. Findings The author first elaborates on Landowska’s “postulates” and then shortly expands on how virtual chatbots such as “AI therapists” pose considerable challenges to emotion recognition algorithms as well. Originality/value This viewpoint’s value is to elaborate and expand on an ongoing discussion on emotion recognition technologies.

Tactile emotion recognition provides a lot of valuable information in human-computer interaction, and it has strong application prospects in many aspects such as smart home and medical treatment. So this situation raises a question: How to quickly and efficiently let the robot perform the correct emotion recognition? In this work, we develop a lifelong learning algorithm which is based on the efficient dictionary learning technology, to tackle the tactile emotion recognition across different tasks. To (...) verify the efficiency of the proposed method, we applied it to two data sets for experimentation: Corpus of Social Touch (CoST) and our dataset(We built it with a 12X12 array sensor). The results show that the proposed lifelong learning algorithm achieves satisfactory results. (shrink)

Financial market and economic growth and development trends can be regarded as an extremely complex system, and the in-depth study and prediction of this complex system has always been the focus of attention of economists and other scholars. Emotion recognition algorithm is a pattern recognition technology that integrates a number of emerging science and technology, and has good non-linear system fitting capabilities. However, using emotion recognition algorithm models to analyze and predict financial market and economic (...) growth and development trends can yield more accurate prediction results. This article first gives a detailed introduction to the existing financial development and economic growth status and development trend forecasting problems, and then gives a brief overview of the concept of emotion recognition algorithms. Then, it describes the emotion recognition methods, including statistical emotion recognition methods, mixed emotion recognition methods, and emotion recognition methods based on knowledge technology, and conducts in-depth research on the three algorithm models of statistical emotion recognition methods, they are the support vector machine algorithm model, the artificial neural network algorithm model, and the long and short-term memory network algorithm model. Finally, these three algorithm models are applied to the financial market and economic growth and development trend prediction experiments. Experimental results show that the average absolute error of the three algorithms is below 25, which verifies that the emotion recognition algorithm has good operability and feasibility for the prediction of financial market and economic growth and development trends. (shrink)

In the human-computer interaction, electroencephalogram access for automatic emotion recognition is an effective way for robot brains to perceive human behavior. In order to improve the accuracy of the emotion recognition, a method of EEG access for emotion recognition based on a deep hybrid network was proposed in this paper. Firstly, the collected EEG was decomposed into four frequency band signals, and the multiscale sample entropy features of each frequency band were extracted. Secondly, the (...) constructed 3D MSE feature matrices were fed into a deep hybrid network for autonomous learning. The deep hybrid network was composed of a continuous convolutional neural network and hidden Markov models. Lastly, HMMs trained with multiple observation sequences were used to replace the artificial neural network classifier in the CNN, and the emotion recognition task was completed by HMM classifiers. The proposed method was applied to the DEAP dataset for emotion recognition experiments, and the average accuracy could achieve 79.77% on arousal, 83.09% on valence, and 81.83% on dominance. Compared with the latest related methods, the accuracy was improved by 0.99% on valence and 14.58% on dominance, which verified the effectiveness of the proposed method. (shrink)

The artificial neural network is employed to study children’s psychological emotion recognition to fully reflect the psychological status of preschool children and promote the healthy growth of preschool children. Specifically, the ANN model is used to construct the human physiological signal measurement platform and emotion recognition platform to measure the human physiological signals in different psychological and emotional states. Finally, the parameter values are analyzed on the emotion recognition platform to identify the children’s psychological (...) and emotional states accurately. The experimental results demonstrate that the recognition ability of children aged 4–6 to recognize the three basic emotions of happiness, calm, and fear increases with age. Besides, there are significant age differences in children’s recognition of happiness, calm, and fear. In addition, the effect of 4-year-old children on the theory of mind tasks is less than that of 5- to 6-year-old children, which may be related to more complex cognitive processes. Preschool children are experiencing a stage of rapid emotional development. If children cannot be guided to reasonably identify and deal with emotions at this stage, their education level and social ability development will be significantly affected. Therefore, this study has significant reference value for preschool children’s emotional recognition and guidance and can promote children’s emotional processing and mental health. (shrink)

One of the significant courses in Chinese universities is English. This course is usually taught by a foreign language instructor. There will, however, necessarily be some communication hurdles between “foreign language teachers” and “native students.” This research presents an emotion recognition method for foreign language teachers in order to eliminate communication barriers between teachers and students and improve student learning efficiency. We discovered four factors of emotion recognition through literature analysis: smile, eye contact, gesture, and tone. (...) We believe that differences in foreign language teachers’ performance in these four areas will have an impact on students’ emotion recognition and, as a result, on their learning efficiency. The influence of the foreign language teacher’s eye contact and gestures is larger in the decision whether can improve the students’ classroom learning efficiency, according to 43 of the questionnaire data analysis. The second is the tone and smile. Our research contributes to the body of knowledge on emotion recognition in university foreign language teachers by presenting a practical method for recognizing emotion in foreign language teachers. We recommend that college foreign language teachers pay attention to eye and gesture communication with students in English classrooms based on the findings. By enriching the style of emotional expression in class, college language teachers, particularly foreign language teachers, can improve communication, and connection with students. (shrink)

There exists substantial cultural variation in how emotions are expressed, but there is also considerable evidence for universal properties in facial and vocal affective expressions. This is the first empirical effort examining the perception of vocal emotional expressions across cultures with little common exposure to sources of emotion stimuli, such as mass media. Shuar hunter-horticulturalists from Amazonian Ecuador were able to reliably identify happy, angry, fearful and sad vocalizations produced by American native English speakers by matching emotional spoken utterances (...) to emotional expressions portrayed in pictured faces. The Shuar performed similarly to English speakers who heard the same utterances in a content-filtered condition. These data support the hypothesis that vocal emotional expressions of basic affective categories manifest themselves in similar ways across quite disparate cultures. (shrink)

Purpose Uncertainty is an under-respected issue when it comes to automatic assessment of human emotion by machines. The purpose of this paper is to highlight the existent approaches towards such measurement of uncertainty, and identify further research need. Design/methodology/approach The discussion is based on a literature review. Findings Technical solutions towards measurement of uncertainty in automatic emotion recognition exist but need to be extended to respect a range of so far underrepresented sources of uncertainty. These then need (...) to be integrated into systems available to general users. Research limitations/implications Not all sources of uncertainty in automatic emotion recognition including emotion representation and annotation can be touched upon in this communication. Practical implications AER systems shall be enhanced by more meaningful and complete information provision on the uncertainty underlying their estimates. Limitations of their applicability should be communicated to users. Social implications Users of automatic emotion recognition technology will become aware of their limitations, potentially leading to a fairer usage in crucial application context. Originality/value There is no previous discussion including the technical view point on extended uncertainty measurement in automatic emotion recognition. (shrink)

Despite the lower levels of psychopathy in women than in men, the scientific interest in studying psychopathy in female participants is increasing. Nevertheless, the number of studies investigating psychopathy in women and associated phenomena remains low. The influence of psychopathy in women inmates on experimental tasks of emotional recognition and moral judgment was evaluated, aiming to contribute to this field of research. Utilitarian moral judgment was predicted by psychopathy, specifically by primary and secondary psychopathy, while primary psychopathy predicted a (...) worse performance on the emotional recognition task. There was no significant influence of general intellectual abilities on either task. Contrarily to what was expected, emotional recognition did not prove to be a significant mediator of the relationship between psychopathy and utilitarian moral judgment. These results emphasize that the tendency to utilitarian moral judgment and worse recognition of facial expressions of emotion are associated to higher psychopathy scores in female inmates (especially primary psychopathy), but more studies are necessary to address the role of the emotional component in the process of moral judgment. (shrink)

Emotion recognition technology uses artificial intelligence to make inferences about a person’s emotions, on the basis of their facial expressions, body language, tone of voice, or other types of input. Underlying such technology are a variety of assumptions about the manifestation, nature, and value of emotions. To assure the quality and desirability of emotion recognition technology, it is important to critically assess the assumptions embedded in the technology. Within philosophy, there is a long tradition of epistemological, (...) ontological, phenomenological, and ethical reflection on the manifestation, nature, and value of emotions. This article draws from this tradition of philosophy of emotions, in order to challenge the assumptions underlying current emotion recognition technology and to promote a more critical engagement with the concept of emotions in the tech-industry. (shrink)

The present study examines the association between people’s interoceptive representation of physical sensations and the recognition of vocal and facial expressions of emotion. We used body maps to study the granularity of the interoceptive conceptualisation of 11 positive emotions (amusement, awe, compassion, contentment, desire, love, joy, interest, pride, relief, and triumph) and a new emotion recognition test (Emotion Expression Understanding Test) to assess the ability to recognise emotions from vocal and facial behaviour. Overall, we found (...) evidence for distinct interoceptive conceptualizations of 11 positive emotions across Asian American, European American, and Latino/a American cultures, as well as the reliable identification of emotion in facial and vocal expressions. Central to new theorising about emotion-related representation, the granularity of physical sensations did not covary with emotion recognition accuracy, suggesting that two kinds of emotion conceptualisation processes might be distinct. (shrink)

According to simulation or shared-substrates models of emotion recognition, our ability to recognize the emotions expressed by other individuals relies, at least in part, on processes that internally simulate the same emotional state in ourselves. The term “emotional expressions” is nearly synonymous, in many people's minds, with facial expressions of emotion. However, vocal prosody and whole-body cues also convey emotional information. What is the relationship between these various channels of emotional communication? We first briefly review simulation models (...) of emotion recognition, and then discuss neuroscientific evidence related to these models, including studies using facial expressions, whole-body cues, and vocal prosody. We conclude by discussing these data in the context of simulation and shared-substrates models of emotion recognition. (shrink)

Purpose The purpose of this paper is to explore uncertainty inherent in emotion recognition technologies and the consequences resulting from that phenomenon. Design/methodology/approach The paper is a general overview of the concept; however, it is based on a meta-analysis of multiple experimental and observational studies performed over the past couple of years. Findings The main finding of the paper might be summarized as follows: there is uncertainty inherent in emotion recognition technologies, and the phenomenon is not (...) expressed enough, not addressed enough and unknown by the users of the technology. Practical implications Practical implications of the study are formulated as postulates for the developers, users and researchers dealing with the technologies of automatic emotion recognition. Social implications As technologies that recognize emotions are becoming more and more common, and perhaps more decisions influencing people lives are to come in the next decades, the trustworthiness of the technology is important from a scientific, practical and ethical point of view. Originality/value Studying uncertainty of emotion recognition technologies is a novel approach and is not explored from such a broad perspective before. (shrink)

Recent studies of emotion mindreading reveal that for three emotions, fear, disgust, and anger, deficits in face-based recognition are paired with deficits in the production of the same emotion. What type of mindreading process would explain this pattern of paired deficits? The simulation approach and the theorizing approach are examined to determine their compatibility with the existing evidence. We conclude that the simulation approach offers the best explanation of the data. What computational steps might be used, however, (...) in simulation-style emotion detection? Four alternative models are explored: a generate-and-test model, a reverse simulation model, a variant of the reverse simulation model that employs an “as if” loop, and an unmediated resonance model. (shrink)

As an important component to promote the development of affective brain–computer interfaces, the study of emotion recognition based on electroencephalography (EEG) has encountered a difficult challenge; the distribution of EEG data changes among different subjects and at different time periods. Domain adaptation methods can effectively alleviate the generalization problem of EEG emotion recognition models. However, most of them treat multiple source domains, with significantly different distributions, as one single source domain, and only adapt the cross-domain marginal (...) distribution while ignoring the joint distribution difference between the domains. To gain the advantages of multiple source distributions, and better match the distributions of the source and target domains, this paper proposes a novel multi-source joint domain adaptation (MSJDA) network. We first map all domains to a shared feature space and then align the joint distributions of the further extracted private representations and the corresponding classification predictions for each pair of source and target domains. Extensive cross-subject and cross-session experiments on the benchmark dataset, SEED, demonstrate the effectiveness of the proposed model, where more significant classification results are obtained on the more difficult cross-subject emotion recognition task. (shrink)

Parkinson´s disease (PD) is a neurodegenerative disorder characterized by motor symptoms following dopaminergic depletion in the substantia nigra. Besides motor impairments however, several non-motor detriments can have the potential to considerably impact subjectively perceived quality of life in patients. Particularly emotion recognition of facial expressions has been shown to be affected in PD, and especially the perception of negative emotions like fear, anger or disgust is impaired. While emotion processing generally refers to automatic implicit as well as (...) conscious explicit processing, the focus of most previous studies in PD was on explicit recognition of emotions only, while largely ignoring implicit processing deficits. Deep brain stimulation of the subthalamic nucleus (STN-DBS) is widely accepted as therapeutic measure in the treatment of PD and has been shown to advantageously influence motor problems. Among various concomitant non-motor effects of STN-DBS, modulation of facial emotion recognition under subthalamic stimulation has been investigated in previous studies with rather heterogeneous results. While there seems to be consensus regarding the processing of disgust which significantly deteriorates under STN stimulation, findings concerning emotions like fear or happiness report heterogeneous data and seem to depend on various experimental settings and measurements. In the present review, we summarized previous investigations focusing on STN-DBS influence on recognition of facial emotional expressions in patients suffering from PD. In a first step, we provide a synopsis of disturbances and problems in facial emotion processing observed in patients with PD. Secondly, we present findings of STN-DBS influence on facial emotion recognition and especially highlight different impacts of stimulation on implicit and explicit emotional processing. (shrink)

With regard to facial emotion recognition, previous studies found that specific facial regions were attended more in order to identify certain emotions. We investigated whether a preferential search for emotion-specific diagnostic regions could contribute toward the accurate recognition of facial emotions. Twenty-three neurotypical adults performed an emotion recognition task using six basic emotions: anger, disgust, fear, happiness, sadness, and surprise. The participants’ exploration patterns for the faces were measured using the Moving Window Technique (MWT). (...) This technique presented a small window on a blurred face, and the participants explored the face stimuli through a mouse-controlled window in order to recognize the emotions on the face. Our results revealed that when the participants explored the diagnostic regions for each emotion more frequently, the correct recognition of the emotions occurred at a faster rate. To the best of our knowledge, this current study is the first to present evidence that an exploration of emotion-specific diagnostic regions can predict the reaction time of accurate emotion recognition among neurotypical adults. Such findings can be further applied in the evaluation and/or training (regarding emotion recognition functions) of both typically and atypically developing children with emotion recognition difficulties. (shrink)

Emotion recognition, and in particular acial emotion recognition (FER), is among the most controversial applications of machine learning, not least because of its ethical implications for human subjects. In this article, we address the controversial conjecture that machines can read emotions from our facial expressions by asking whether this task can be performed reliably. This means, rather than considering the potential harms or scientific soundness of facial emotion recognition systems, focusing on the reliability of (...) the ground truths used to develop emotion recognition systems, assessing how well different human observers agree on the emotions they detect in subjects’ faces. Additionally, we discuss the extent to which sharing context can help observers agree on the emotions they perceive on subjects’ faces. Briefly, we demonstrate that when large and heterogeneous samples of observers are involved, the task of emotion detection from static images crumbles into inconsistency. We thus reveal that any endeavour to understand human behaviour from large sets of labelled patterns is over-ambitious, even if it were technically feasible. We conclude that we cannot speak of actual accuracy for facial emotion recognition systems for any practical purposes. (shrink)

Ambient Assisted Living (AAL) refers to technologies that track daily activities of persons in need of care to enhance their autonomy and minimise their need for assistance. New technological developments show an increasing effort to integrate automated emotion recognition and regulation (ERR) into AAL systems. These technologies aim to recognise emotions via different sensors and, eventually, to regulate emotions defined as “negative” via different forms of intervention. Although these technologies are already implemented in other areas, AAL stands out (...) by its tendency to enable an inconspicuous 24-hour surveillance in the private living space of users who rely on the technology to maintain a certain degree of independence in their daily activities. The combination of both technologies represents a new dimension of emotion recognition in a potentially vulnerable group of users. Our paper aims to provide an ethical contextualisation of the novel combination of both technologies. We discuss different concepts of emotions, namely Basic Emotion Theory (BET) and the Circumplex Model of Affect (CMA), that form the basis of ERR and provide an overview over the current technological developments in AAL. We highlight four ethical issues that specifically arise in the context of ERR in AAL systems, namely concerns regarding (1) the reductionist view of emotions, (2) solutionism as an underlying assumption of these technologies, (3) the privacy and autonomy of users and their emotions, (4) the tendency of machine learning techniques to normalise and generalise human behaviour and emotional reactions. (shrink)

Emotion recognition plays an important role in intelligent human–computer interaction, but the related research still faces the problems of low accuracy and subject dependence. In this paper, an open-source software toolbox called MindLink-Eumpy is developed to recognize emotions by integrating electroencephalogram and facial expression information. MindLink-Eumpy first applies a series of tools to automatically obtain physiological data from subjects and then analyzes the obtained facial expression data and EEG data, respectively, and finally fuses the two different signals at (...) a decision level. In the detection of facial expressions, the algorithm used by MindLink-Eumpy is a multitask convolutional neural network based on transfer learning technique. In the detection of EEG, MindLink-Eumpy provides two algorithms, including a subject-dependent model based on support vector machine and a subject-independent model based on long short-term memory network. In the decision-level fusion, weight enumerator and AdaBoost technique are applied to combine the predictions of SVM and CNN. We conducted two offline experiments on the Database for Emotion Analysis Using Physiological Signals dataset and the Multimodal Database for Affect Recognition and Implicit Tagging dataset, respectively, and conducted an online experiment on 15 healthy subjects. The results show that multimodal methods outperform single-modal methods in both offline and online experiments. In the subject-dependent condition, the multimodal method achieved an accuracy of 71.00% in the valence dimension and an accuracy of 72.14% in the arousal dimension. In the subject-independent condition, the LSTM-based method achieved an accuracy of 78.56% in the valence dimension and an accuracy of 77.22% in the arousal dimension. The feasibility and efficiency of MindLink-Eumpy for emotion recognition is thus demonstrated. (shrink)

With the development of modern information technology, the flipped classroom teaching mode came into being. It has gradually become one of the hotspots of contemporary educational circles and has been applied to various disciplines at the same time. The domestic research on the flipped classroom teaching mode is still in the exploratory stage. The application of flipped classroom teaching mode is still in the exploratory stage. It also has many problems, such as low class efficiency, poor teacher-student interaction, outdated teaching (...) modes, not student-centered, etc., which lead to poor students’ enthusiasm for learning. Therefore, the current English flipped classroom teaching mode still needs to be tested and revised in practice. Combined with emotion recognition technology, this paper analyzes speech emotion recognition, image emotion recognition, and audition emotion recognition technology and conducts a revision test for the current English flipped classroom teaching mode. It uses the SVM algorithm for one-to-one method and dimension discretization for emotion recognition, and finds that the recognition results after different dimension classification recognition are improved for each emotion. Among them, the recognition rate of different dimension classification recognition methods is 2.6% higher than that of one-to-one method. This shows that under the same conditions, the emotion recognition technology of different dimension classification recognition methods is higher. (shrink)

ABSTRACTPrevious research on the development of emotion recognition in music has focused on classical, rather than popular music. Such research does not consider the impact of lyrics on judgements...

ABSTRACTRecently, Cognition and Emotion published an article demonstrating that age cues affect the speed and accuracy of emotion recognition. The authors claimed that the observed effect of target age on emotion recognition is better explained by evaluative than stereotype associations. Although we agree with their conclusion, we believe that with the research method the authors employed, it was impossible to detect a stereotype effect to begin with. In the current research, we successfully replicate previous findings. (...) Furthermore, by changing the comparative context, Study 2 provides a first test of age-stereotypes affecting emotions recognition. We discuss recommendations for future studies in the domain of social categorisation and emotion recognition. (shrink)

This exploration aims to study the emotion recognition of speech and graphic visualization of expressions of learners under the intelligent learning environment of the Internet. After comparing the performance of several neural network algorithms related to deep learning, an improved convolution neural network-Bi-directional Long Short-Term Memory algorithm is proposed, and a simulation experiment is conducted to verify the performance of this algorithm. The experimental results indicate that the Accuracy of CNN-BiLSTM algorithm reported here reaches 98.75%, which is at (...) least 3.15% higher than that of other algorithms. Besides, the Recall is at least 7.13% higher than that of other algorithms, and the recognition rate is not less than 90%. Evidently, the improved CNN-BiLSTM algorithm can achieve good recognition results, and provide significant experimental reference for research on learners’ emotion recognition and graphic visualization of expressions in an intelligent learning environment. (shrink)

Theories on empathy have argued that feeling empathy for others is related to accurate recognition of their emotions. Previous research that tested this assumption, however, has reported inconsiste...