Spectators, a joy to watch

Projektstart: 01.01.2009
Projektträger: Universität Augsburg
Projektverantwortung vor Ort: Prof. Dr. Elisabeth André


spectators_logo"Spectators, a joy to watch" is an application of virtual spectators that react in real-time on events of a tennis match. The virtual spectators adapt their facial expression, body posture and gestures according to their current emotions dependent on their favorite in the match.


The Scene

There is a handful of spectators that are really excited about the game. The spectators have different preferences about the teams that they favor or are neutral. To vary the mood-states, each spectator has an euphoria factor that determines how much a change in the mood-state will affect it. The mood-state of a spectator is shown with expressive facial expressions and some typical animations. Additionally to the animations and expressions the mood-state is expressed by speech as well.
The moods that are expressed are euphoric, happy, slightly happy, neutral, slightly sad, sad and disappointed. The division of these fields is not uniformly distributed. The neutral field is bigger than others so that not every score will result in a change of the mood-state if it is currently neutral. The fields for the extreme values euphoric and disappointed is quite small so that even a simple point may result in a change. Another factor that results the mood-state is the importance of the scene. The effect of a score in a match ball is much higher than a 15-15. From time to time, a spectator will not just look at the ball, but look at a specic player. This results in a behavior variant in the spectator crowd and thus in a more natural appearance of the scene. To make the scene more convincing, a referee is integrated that comments the current score via text-to-speech as you know it from TV.

The System

Parsing and simulating events

A major part of this work is to interpolate the position values of the ball between the given positions. This interpolation is necessary because otherwise the spectators would not be able to follow the ball.

Inverse kinematics

The gaze behavior of the spectators is done via inverse kinematics. Inverse kinematics is the process of determining the parameters of a jointed flexible object (a kinematic chain) in order to achieve a desired pose. The GameIKComponent, which was developed for the Horde3D GameEngine, uses the so called Cyclic-Coordinate Descent (CCD) method. This method has many advantages, one of them being the ease with which one can extend its range of use beyond simple IK. Another functionality of the GameIKComponent, called IK Gaze, allows moving the eyes and head of a character. It is based on an adjusted version of the CCD algorithm. The gaze problematic can be viewed as a more specific case of inverse kinematics, and therefor treated in a similar way by manipulating a predened kinematic chain until the last node of the chain (called end effector) points towards the target, in this case, until the eyes look at the target. The fact that it uses an iterative algorithm with simple loops makes IK Gaze a reliable, real-time method of animating characters. IK Gaze gives us a realistic way of porting the human gaze action on virtual characters. Emotions, conversations and other social behaviors become more human-like.


The spectators

As our Tennis Spectator, we chose a cartoon character called Ritchie, of whom an older version has already been used for various projects. The new model uses the Facial Action Coding System (FACS) for facial animation, which was developed by Ekman and Friesen. In FACS, expressions are split into Action Units (AU) which mostly correspond to the movement of a particular face muscle. By recombining these Action Units, complex emotional and conversational signals are created. This method allows for more flexible and predictable animation than blending between basic emotions of the whole face, since all parts are assigned individual deformations. Consequently, possible conflicts between contrary movements (such as a smiling mouth speaking an O) canbbe avoided by choosing either of them or fine-tuning their respective intensity. An even more important benefit is that new, more specific facial expressions can be easily composed based on parameters. The Ritchie model was created using Autodesk's 3ds Max, with Biped animations for gestures and Morph Targets for the required AUs. These were created consulting mainly the Facial Expression Repertoire, which provides video examples for almost all AUs, as well as a library showing how to combine them to produce a wide range of emotional expressions.