Dr. Gregor Ulrich Mehlmann

M.Sc. Gregor Ulrich Mehlmann


Telefon: 0049 - 821 - 598 - 2331
Telefon (weitere): 0049 - 821 - 598 - 2341
Fax: 0049 - 821 - 598 - 2349
Raum: Building N, Room 2040
Hausanschrift: Universitäts-Straße 6a
D - 86159 - Augsburg


I have successfully finished my PhD thesis and am now with the TriCAT GmbH in Ulm. Before I finished my PhD, I was employed as a developer, researcher and lecturer at the laboratory for Human Centered Multimedia, located at the Augsburg University in Germany from 2010 until 2018. Before I went to Augsburg I was employed as a research associate at the German Research Center for Artificial Intelligence (DFKI). I received a Bachelor degree in computer science with a specialization in computer graphics and software engineering from the Saarland University and a Honor's Degree as Master of computer science with the specialization in artificial intelligence at the DFKI and the Saarland University.


My research interests lie in the exploration and application of formal methods from artificial intelligence as well as models from cognitive sciences for the development of intelligent systems. My main research area is the development of intelligent multi-modal user interfaces for human computer interaction and the modeling of natural behavior and dialog for embodied conversational agents and social collaborative robots in interactive applications used for entertainment, training, health-care, elder-care and education.



      Multimedia II Lectures:

      Multimodal UI Projects:

      Multimedia Project Topics:

      Master & Bachelor Seminars:

      Game Programming Lectures:


Some theses of M.Sc and B.Sc students that I have been supervising or co-supervising or that I have been involved in as an advisor or consultant:

  • Master Thesis - Modeling Engagement, Overlaps and Barge-In Behavior in Human-Agent-Interaction
  • Project Module - Modeling Bidirectional Gaze Behaviors for Grounding in Human Robot Interaction
  • Master Thesis - Modeling Multimodal Engagement Recognition in Human-Agent-Interaction Using Event Logic Charts
  • Master Thesis - An ANVIL-Plugin for Multimodal Event Simulation and Benchmarking Interaction Models
  • Master Thesis - Situative Sprach-, Blick- und Zeigegesten-Basierte Mensch-Roboter Interaktion
  • Master Thesis - Augmenting Statecharts with a Temporal Multimodal Event Logic for Multimodal Integration
  • Master Project - Anbindung der Visual SceneMaker Software an den Reeti Roboter
  • Master Project - Simulations-Basierte Validierung von Interaktionsmodellen Basierend auf der ANVIL Software
  • Bachelor Thesis - Entwicklung einer Multi-Modalen Multi-Display Anwendung zur Kartenbasierten Suche
  • Bachelor Thesis - Plattformübergreifende Visualisierung von FACS Gesichtsausdrücken Virtueller Charaktere
  • Bachelor Thesis - Modeling Dialog and Interaction Management for Interactive Storytelling Applications
  • Bachelor Thesis - Augmenting the SSI Framework with Real-Time Object Detection on Eye-Tracking Data
  • Bachelor Thesis - A Case-based and Context-Aware Recommender System for Ambient Assisted Living Scenarios
  • Bachelor Thesis - Ontologie-Basierte Modellierung eines Benutzermodells für den Einsatz in AAL-Systemen
  • Bachelor Thesis - Modelierung von Dialogorientiertem Blickverhalten für Soziale Humanoide Roboter
  • Bachelor Project - Modellierung von Verhaltensaspekten Virtueller Charaktere in Dialogen
  • Bachelor Project - Modeling Virtual Characters' Different Gaze Strategies During Dialogs
  • Bachelor Project - Portierung des Virtuellen Gesprächspartners Alfred als auf ein Smartphone


Research projects that I have been involved in as researcher or advisor:

EmpaT Empathische Trainingsbegleiter für den Bewerbungsprozess.
EmpaT is a BMBF founded research project, in which we develop an empathic virtual character for a training system that is supposed to help the users to practice their social skills for real job interview situations in order to be able to better analyze and rate their behavior and performance in such stressful situations.
KRISTINA A Knowledge-Based Information Agent with Social Competence and Human Interaction Capabilities.
Kristina is an EU funded research project, which aims at researching and developing technologies for a human-like socially competent and communicative agent that is run on mobile communication devices and that serves for migrants with language and cultural barriers in the host country.
TARDIS Training Young Adult's Regulation of Emotions and Development of Social Interaction Skills.
TARDIS was an EU funded research project, in which we developed a virtual character that plays the role of an interactive virtual job recruiter that helps young unemployed or uneducated people to practice their socio-emotional skills and stress management in simulated job interviews.
DynaLearn Engaging and Informed Tools for Learning Conceptual System Knowledge.
DynaLearn was an EU funded research project, in which we created an interactive learning environment for conceptual modeling and provided a full set of animated virtual characters that not only supported the users in meaning of help agents, but that also reflected the user's knowledge.
eCute Technology Enhanced Education in Cultural Understanding.
eCUTE was an EU funded research project, in which we aimed at improving the understanding between different cultural, ethnic and religious groups by developing several cultural virtual learning environments for late-primary children and young adults based on virtual dramas using synthetic characters with culturally-specific interaction behavior.
IRIS Integrating Research in Interactive Digital Storytelling.
IRIS aimed at creating a virtual center of excellence that would be able to achieve breakthroughs in the understanding of interactive digital storytelling and the development of corresponding technologies. A major endeavor was to develop new media which could offer a radically new user experience, with a potential to revolutionize digital entertainment.


Software projects that I have been involved in as developer or advisor:

VisualSceneMaker³ The VSM framework is an authoring and modeling tool for creating interactive presentations aimed to non-programming experts. It supports the modeling of verbal and non-verbal behavior of Virtual Characters and robots. Therefore, it provides users with a graphical interface and a simple scripting language that allows them to create rich and compelling content.
The SSI Framework The SSI framework offers tools to record, analyze and recognize human behavior in real-time, such as gestures, mimics, head nods, and emotional speech. In particularly SSI supports the machine learning pipeline in its full length and offers a graphical interface that assists a user to collect own training corpora and obtain personalized models. I have contributed to the SSI framework by implementing a plug-in that integrates the SMI eye-tracking glasses into SSI and is able to compute probability distributions for the user's fixated objects which are recognized using the ARTK toolkit.
Horde3D Game Engine The H3D game engine is based on the Horde3D graphics engine which offers stunning visual effects expected in next-generation games. At the same time it is as lightweight and conceptually clean as possible.
Advanced Agent Animation Advanced Agent Animation (AAA) is a component of the Horde3D GameEngine which is based on the Horde3D rendering engine. It is designed for creating virtual environments and scenarios in which it is essential to simulate social interactions with multiple virtual characters. The application interface of AAA provides extended support for manipulating and customizing the autonomous behavior of virtual characters and managing social interactions among them. Each agent in the scene is able to perform , Text-To-Speach, gestures, postures, gaze behaviors, movements as well as different kinds of positioning and orientation in multi-agent conversations.


Conferences and journals for which I have been reviewer or member of the PC:

  • AGNT - Journal of Autonomous Agents and Multi-Agent Systems
  • IJCAI - International Joint Conference on Artificial Intelligence
  • ICMI - International Conference on Multimodal interaction
  • ICSR - International Conference on Social Robotics
  • CASA - International Conference on Computer Animation and Social Agents
  • EICS - ACM SIGCHI Symposium on Engineering Interactive Computing Systems
  • CHI - ACM SIGCHI Conference on Human Factors in Computing Systems


Conferences and journal papers that I have published:

  • Gregor Mehlmann, Kathrin Janowski and Elisabeth André. Modeling Grounding for Interactive Social Companions. Article in the Journal KI - Künstliche Intelligenz, German Journal on Artificial Intelligence, Technical Contribution in the Special Issue on Social Companion Technologies, September 2015, pages 1 - 8, Springer-Verlag Berlin Heidelberg.
  • Tobias Baur, Gregor Mehlmann, Ionut Damian, Patrick Gebhard, Florian Lingenfelser, Johannes Wagner, Birgit Lugrin, and Elisabeth André. Context-Aware Automated Analysis and Annotation of Social Human--Agent Interactions. Article in the ACM Transactions on Interactive Intelligent Systems, TiiS, Special Issue on Behavior Understanding for Arts and Entertainment (Part 1 of 2) archive, July 2015, Volume 5, Issue 2, Article No. 11, ACM New York, NY, USA, 2015.
  • Ionut Damian, Tobias Baur, Birgit Lugrin, Patrick Gebhard, Gregor Mehlmann, Elisabeth André. Games are Better than Books: In-Situ Comparison of an Interactive Job Interview Game with Conventional Training. In Proceedings of the 17th International Conference on Artificial Intelligence in Education, AIED 2015, Madrid, Spain, June 22-26, 2015, Volume 9112 of the Series Lecture Notes in Computer Science, pages 84-94, Springer International Publishing Switzerland.

  • Birgit Endrass, Christoph Klimmt, Gregor Mehlmann, Elisabeth André, Christian Roth. Designing User-Character Dialog in Interactive Narratives: An Exploratory Experiment. Article in the IEEE Transactions on Computational Intelligence and AI in Games, Special Issue on Computational Narrative and Games, June 2014, Volume 6, Issue 2, pages 166 - 173, IEEE Computational Intelligence Society.
  • Patrick Gebhard, Tobias Baur, Ionut Damian, Gregor Mehlmann, Johannes Wagner, Elisabeth André. Exploring Interaction Strategies for Virtual Characters to Induce Stress in Simulated Job Interviews. In Alessio Lomuscio, Paul Scerri, Ana Bazzan, and Michael Huhns, editors, Proceedings of the 13th International Conference on Autonomous Agents and Multiagent Systems, AAMAS2014, Paris, France, May 5-9, 2014, pages 661-668, International Foundation for Autonomous Agents and Multi-agent Systems.
  • Gregor Mehlmann, Kathrin Janowski, Tobias Baur, Elisabeth André Markus Haering, and Patrick Gebhard. Exploring a Model of Gaze for Grounding in HRI. In Proceedings of the 16th ACM International Conference on Multimodal Interaction, ICMI 2014, Istanbul, Turkey, November 12-16, 2014, ACM International Conference Proceedings, pages 247-254, ACM, New York, NY, USA, 2014.
  • Gregor Mehlmann, Kathrin Janowski, Tobias Baur, Elisabeth André Markus Haering, and Patrick Gebhard. Modeling Gaze Mechanisms for Grounding in HRI. In Proceedings of the 21th European Conference on Artificial Intelligence, ECAI 2014, Prague, Czech Republic, August 18-22, 2014, Frontiers in Artificial Intelligence and Applications, pages 1069–1070, IOS Press E-books, Amsterdam, The Netherlands, 2014.

  • Patrick Gebhard, Gregor Mehlmann, and Michael Kipp. Visual Scenemaker - A Tool for Authoring Interactive Virtual Characters. Article in the Journal on Multimodal User Interfaces, JMUI, Special Issue on Interacting with Embodied Conversational Agents, Springer-Verlag, July 2012, Volume 6, Issue 1, pages 3-11, Springer-Verlag, 2012.
  • Gregor Mehlmann and Elisabeth André. Modeling Multimodal Integration with Event Logic Charts. In Proceedings of the 14th ACM International Conference on Multimodal Interaction, ICMI 2012, Santa Monica, California, USA, October 22-26, 2012, ACM International Conference Proceedings, pages 125–132. ACM, New York, NY, USA, 2012.
  • Michael Wißner, Wouter Beek, Esther Lozano, Gregor Mehlmann, Floris Linnebank, Jochem Liem, Markus Häring, René Bühling, Jorge Gracia, Bert Bredeweg, Elisabeth André. Increasing Learners' Motivation through Pedagogical Agents: The Cast of Virtual Characters in the DynaLearn ILE. In Proceedings of the International Workshop on Agents for Educational Games and Simulations, AEGS 2011, Taipei, Taiwan, May 2, 2011, Volume 7471 of the series Lecture Notes in Computer Science pages 151-165, Springer-Verlag Berlin Heidelberg.

  • Gregor Mehlmann, Patrick Gebhard, Birgit Endrass, and Elisabeth André. Scenemaker: Visual Authoring of Dialog Processes. In Proceedings of the 7th Workshop on Knowledge and Reasoning in Practical Dialog Systems, KRPDS 2011, held at the 22th International Joint Conference on Artificial Intelligence, IJCAI 2011, Barcelona, Spain, July 16-22, 2011, pages 24–36.
  • Birgit Endrass, Nick Degens, Gert Jan Hofstede, Elisabeth André, John Hodgson, Samuel Mascarenhas, Gregor Mehlmann, Ana Paiva, Christopher Ritter, and Aleksandra Swiderska. Integration and Evaluation of Prototypical Culture-related Differences. In Proceedings of the Workshop on Culturally Motivated Virtual Characters, CMVC 2011, held at the 11th International Conference on Intelligent Virtual Agents, IVA 2011, Rejkjavik, Iceland, September 15-17, 2011.
  • Birgit Endrass, Christoph Klimmt, Gregor Mehlmann, Elisabeth André, Christian Roth. Exploration of User Reactions to Different Dialog-Based Interaction Styles. In Proceedings of the 4th International Conference on Interactive Digital Storytelling, ICIDS 2011, Vancouver, Canada, November 28 – 1 December, 2011, Volume 7069 of the series Lecture Notes in Computer Science, pages 243-248, Springer-Verlag GmbH Berlin Heidelberg.
  • Gregor Mehlmann, Birgit Endrass, and Elisabeth André. Modeling Parallel State Charts for Multithreaded Multimodal Dialogs. In Proceedings of the 13th International Conference on Multimodal Interaction, ICMI 2011, Alicante, Spain, November 14-18, 2011, ACM International Conference Proceedings, pages 385–392. ACM, New York, NY, USA, 2011.
  • Michael Wissner, Wouter Beek, Esther Lozano, Gregor Mehlmann, Floris Linnebank, Jochem Liem, Markus Häring, René Bühling, Jorge Gracia, Bert Bredeweg, Elisabeth André. Character Roles and Interaction in the DynaLearn Intelligent Learning Environment. In Proceedings of the 15th International Conference, AIED 2011, Auckland, New Zealand, June 28 – July 2011, Volume 6738 of the series Lecture Notes in Computer Science, pages 585-587, Springer-Verlag GmbH Berlin Heidelberg.

  • Birgit Endrass, Michael Wißner, Gregor Mehlmann, René Bühling, Markus Häring, and Elisabeth André. Teenage Girls as Authors for Digital Storytelling - A Practical Experience Report. In Workshop on Education in Interactive Digital Storytelling held on the 3rd International Conference on Interactive Digital Storytelling, ICIDS 2010, Edinburgh, UK, November 1 - 3, 2010.
  • Gregor Mehlmann, Markus Haering, René Buehling, Michael Wissner, and Elisabeth André. Multiple Agent Roles in an Adaptive Virtual Classroom Environment. In Jan Allbeck, Norman Badler, Timothy Bickmore, Catherine Pelachaud, and Alla Safonova, editors, Proceedings of the 10th International Conference on Intelligent Virtual Agents, IVA 2011, Philadelphia, PA, USA, September 20-22, 2010, volume 6356 of Lecture Notes in Computer Science, pages 250–256. Springer Berlin Heidelberg, 2010.
  • René Buehling, Michael Wissner, Markus Haering, Gregor Mehlmann and Elisabeth André. Design Decisions for Virtual Characters in the DynaLearn Interactive Learning Environment. Chapter in the Book of Abstracts of the 7th International Conference on Ecological Informatics, ISEI 7, Ghent, Belgium, December 13 - 16, 2010, pages 144 - 145.