Multimodal emotion recognition from low-level cues

Maja Pantic, George Caridakis, Elisabeth André, Jonghwa Kim, Kostas Karpouzis and Stefanos Kollias

erschienen 2011 "Emotion-Oriented Systems", Pages 115 - 132

Verlag: Springer

ISBN: 978-3-642-15183-5


Emotional intelligence is an indispensable facet of human intelligence and one of the most important factors for a successful social life. Endowing machines with this kind of intelligence towards affective human–machine interaction, however, is not an easy task. It becomes more complex with the fact that human beings use several modalities jointly to interpret affective states, since emotion affects almost all modes – audio-visual (facial expression, voice, gesture, posture, etc.), physiological (respiration, skin temperature, etc.), and contextual (goal, preference, environment, social situation, etc.) states. Compared to common unimodal approaches, many specific problems arise from the case of multimodal emotion recognition, especially concerning fusion architecture of the multimodal information. In this chapter, we firstly give a short review for the problems and then present research results of various multimodal architectures based on combined analysis of facial expression, speech, and physiological signals. Lastly we introduce designing of an adaptive neural network classifier that is capable of deciding the necessity of adaptation process in respect of environmental changes.


  • BibTeX  -  (BibTeX.txt, 0 KB)