Avatars used in "Changing Outcomes"

Excerpts from page 64 and 65 of VIRTUAL REALITY FOR PSYCHOTHERAPY: CURRENT REALITY AND FUTURE POSSIBILITIES by KALMAN GLANTZ and ALBERT (SKIP) RIZZO from Psychotherapy: Theory, Research, Practice, Training Copyright 2003 by the Educational Publishing Foundation 2003, Vol. 40, No. 1/2, 55–67

The Future
It is probable that the ultimate potential of VR for therapy lies in the simulation of the social, as opposed to the physical, environment. The challenge . . . is to create and animate virtual personae (avatars)—characters who look, sound, and “behave” like people—in particular, people who are significant in the client’s life and memory (e.g., family members, friends, enemies, colleagues and bosses). (Glantz et al., 1996, p.467). Such avatars could be used in exploratory psychotherapy in a variety of ways: as a tool to promote discovery of repressed material, as a means of evoking past traumas, and/or as a way of “changing outcomes.” Work on the development of such advanced avatars is ongoing but is in the early stages. The problems are legion. Such avatars must have some level of artificial intelligence that allows them to act and react in their simulated environments on the basis of inputs from a multitude of sources. They must exploit the full gamut of natural language processing, including speech recognition and speech generation. They must move in concert with their speech. And they must behave in ways that make sense to humans psychologically. To produce such creatures, experts from a large number of disciplines must collaborate. In the spring of 2002, researchers from across disciplines met at the University of Southern California to organize a framework for research and development. Participants were drawn from astonishingly diverse areas of expertise, including human figure animation, facial animation, artificial intelligence, perception, cognitive modeling, personality and emotion, natural language processing, speech recognition and synthesis, nonverbal communication, distributed simulation, and computer gaming. Their goal, which is not yet close to realization, is to create shareable tools, modular architecture, and interface standards that will allow researchers to build on each other’s work, with the goal of creating simulations that support bidirectional human–avatar interaction. The obstacles are numerous, but the building blocks that will lead to realistic virtual humans are in preparation (see Gratch et al., Glantz, Rizzo, and Graap 2002, for details on some of the more technical aspects of this work). The importance of personalizing images has been highlighted by a team working at Eastman Kodak (Fedorovskaya & Endrikhovski, 2003). This group has been testing the hypothesis that (a) it is possible to predictably change the emotional state of a person by presenting a sequence of images and (b) the effectiveness of the sequence can be enhanced if the images are matched to the personality and/or experience of the participant. Their pilot experiments have shown that individuals have a “pictorial profile”; each individual has a different physiological response to the images. This finding supports the hypothesis that the images selected for therapeutic use need to be individualized. For such clinical purposes, there is a need for avatars with facial expressions that can be read accurately. Another team at the University of Southern California has been working to develop avatars with improved facial expressions (Rizzo, Neumann, Enciso, Fidaleo, and Noh, 2001). Using a Performance Driven Facial Animation (PDFA) system that tracked and captured actual facial movement needed to “fuel” avatar expressions, the team created avatars that expressed six well documented universal human emotions: happiness, sadness, fear, anger, disgust, and surprise. Two other expressions were also captured: attentiveness and puzzlement. The team then compared how well people decoded emotional expression from these avatars with how they rated video clips of the same facial expressions of real people on video clips. The videotapes and three-dimensional avatar animations of the same facial expressions were randomly presented to 38 human raters. The study produced mixed results. Successful decoding of facial expressions varied depending on the emotion expressed. Raters were better able to detect the correct emotion when the avatars were conveying happiness, surprise, and sadness. However, raters had more difficulty correctly detecting disgust, fear, anger, attentiveness, and puzzlement on the faces of the avatar. Although methods for avatar animation have evolved considerably since the time of this study, these results provide significant information as to what is needed to improve the realism and communicability of avatar facial expressions. Most notably, subtle eye and mouth movements that the avatars in this study were incapable of producing were seen to limit comprehensive emotional expression (Rizzo et al., 2001). Greatest progress in developing useful avatars has been made in the area of training, where the avatar serves as an instructor or teammate. One of the most sophisticated applications is a programmable avatar called Steve, the creation of the University of Southern California’s Information Sciences Institute and Institute for Creative Technologies (see Rickel and Johnson; 1999, 2000). Steve already has many capabilities that allow for face-to-face collaboration with people. He can lead students around a virtual world, demonstrate tasks, guide their attention using his gaze and pointing gestures, and play the role of a teammate whose activities can be monitored. Steve’s behavior is not scripted. He can be given knowledge of new tasks and domains and then perform those tasks in those domains, using general, domain independent capabilities. He can also express emotions, a capacity that was generated using psychological theories that emphasize the relationship between emotion, cognition, and behavior. Imagine a “Steve” that can be programmed to look, act, and sound like a client’s father and mother! The possibilities are endless. Scenes can be revisited, rehearsals conducted, memories and fantasies explored, and outcomes changed. The power of psychotherapy would be greatly enhanced. We provide a note of caution, however: such applications still lie in the future. Furthermore,the idea that environments need to be individualized to create therapeutic effectiveness is merely a hypothesis. Standardized environments that are well-designed may turn out to be all that is possible or necessary. In any case, the role of the therapist will remain as essential as it is now. The computer is simply one more tool at the disposition of the healing professional.