Multimodal interaction with a virtual character in interactive storytelling

Nikolaus Bee, Johannes Wagner, Elisabeth André, Fred Charles, David Pizzi and Marc Cavazza

erschienen 2010 "Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems: volume 1 - Volume 1", AAMAS '10, Pages 1535 - 1536

Verlag: International Foundation for Autonomous Agents and Multiagent Systems

ISBN: 978-0-9826571-1-9 DOI:


A number of interactive storytelling (IS) systems offer the user the possibility to input natural language input which determines how a story progresses. We introduce a framework for real-time signal processing (SSI) to analyze the users' state which then can influence the feelings of the story characters and their actions in the story. SSI is not only used to enable more natural character responses that are sensitive to the user's state at runtime. In addition SSI offers the possibility to collect a large variety of synchronized user data which can be used to analyze the user's experience in offline mode. The underlying narrative in which the approach was tested is based on a classical XIXth century psychological novel: Madame Bovary, by Flaubert.