Suche

Modeling and Interpretation of Multithreaded and Multimodal Dialogue

Gregor Mehlmann, Birgit Endrass and Elisabeth André

Modeling and Interpretation of Multithreaded and Multimodal Dialogue

Proceedings of the 13th International Conference on Multimodal Interaction, ICMI' 11, pp. 385-392, Alicante, Spain, November 14-18, 2011


In this paper, we present a modeling approach for the management of highly interactive, multithreaded and multimodal dialogues. Our approach enforces the separation of dialogue content and dialogue structure and is based on a statechart language enfolding concepts for hierarchy, concurrency, variable scoping and a detailed runtime history. These concepts facilitate the modeling of interactive dialogues with multiple virtual characters, autonomous and parallel behaviors, flexible interruption policies, context-sensitive interpretation of the user's discourse acts and coherent resumptions of dialogues. An interpreter allows the realtime visualization and modification of the model to allow a rapid prototyping and easy debugging. Our approach has successfully been used in applications and research projects as well as evaluated in field tests with non-expert authors. We present a demonstrator illustrating our concepts in a social game scenario.