Trickster at the intersection

Trickster at the intersection: exploring virtual theater performance and interaction Pierre Boulanger Computing Science University of Alberta [email protected] Maryia Kazakevich AICT University of Alberta [email protected]

Qiong Wu Computing Science University of Alberta [email protected] Robyn Taylor Computing Science University of Alberta [email protected] ABSTRACT

Trickster at the intersection is a virtual theater system designed to explore audience interaction and involvement in a participatory theater performance. Allowing participants to enact change in the development of a theatrical experience allows them to function not only as users of the creative system, but also as co-creators. We describe how the Trickster installation enables audiences to collaborate with the avatar in order to push forward the narrative, preserving the interactivity and improvisation of the theatrical art form and affording users a better sense of presence in the shared virtual world.

Figure 1. On the performing set, an actor is driving an avatar and its virtual world to push forward the narrative development.

Author Keywords

virtual reality, human computer interaction ACM Classification Keywords

H.5.1 Information Interfaces and Presentation: Multimedia Information Systems—Artificial, augmented, and virtual realities Figure 2. The rendered image of the virtual world.

PROJECT INTRODUCTION

Trickster at the intersection is devised to explicitly explore the relationships between narrative author, performer, and audience, within the context of virtual theater. Virtual theatre allows participant audience members to collaborate with performing artists and digital avatars to experience a story unfolding inside an interactive virtual environment. Using interactive technology to facilitate participation enables participants to transcend their role as audience member from that of a passive spectator to that of an active collaborator, allowing them to take part in the unfolding of the virtual story.

As a new form of digital media, the Trickster system [3] extends previous forms of virtual theater in many ways. Conventional technology that uses virtual reality in a theatrical performance often integrates prerecorded animation, making the play feel stilted, lifeless and unresponsive [2]. We explore a new relationship between actors and audiences in a theatrical set embodied in the virtual world. Integrating perspectives of director, performer and audience, the play designed by the system not only preserves the theatrical performance form through direct mediated interaction of avatars and objects, but also innovates by introducing aspects of flexible interaction amongst different play elements (e.g. audience, actor, avatar, virtual objects etc.) The ability to allow an audience member to also function as participant and improvisational collaborator allows us to explore a novel form of performance that is impossible to realize in a traditionally uni-directional theatrical form.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. CHI 2011, May 7–12, 2011, Vancouver, BC, Canada. Copyright 2011 ACM 978-1-4503-0267-8/11/05...$10.00.

The system’s hardware interface is distributed in two differ1

Figure 3. Pictures taken on the display set at the performance in Banff

performance show by Mateas and Stern, called “interactive drama” [1], we replace the need for standard input devices for a much more expressive, physically natural, and intuitive gesture-based tool that truly allows the performer to create a sense of shared presence between the real and virtual entities co-inhabiting the performance space.

ent remote sets: the performing set, sequestered out of view from the audience (Figure 1) and the display set, which the audience can see and interact with. In the sequestered set, actors perform inside a motion capture environment. Their motion is directly mapped onto the corresponding avatar character in real-time, allowing natural interaction between the actor (who is represented by the avatar) and the virtual world. The theatrical set visible to the audience uses transparent projection sheet technologies 1 to set up an immersive threewall CAVE as an interface for audience members to enter and interact with the virtual performance environment.

The aesthetic content of the play creates a platform in which users can explore their roles in an incremental way. There are different levels of relationship implicitly contained in the play that can entice people to investigate. The first level is between the avatar and virtual objects. Instead of randomly growing plants in different places, the relationship between the behavior animation of virtual object and the avatar gesture are defined, such as allowing objects to appear/disappear, fly, translate, and deform etc. Special postures and gestures performed by the actor are coded in the system to trigger different special effects that are meaningful in the narrative. The second level is between the actor and avatar. Motions of the avatar mimic the actor’s motion in real time. Viewing the avatar as an intermediate entity that is mediated by the actor links the actors experience more closely to the events in the virtual world. The third level is between the avatar and audience. By interacting with the avatar, audience is involved in the co-creation of the virtual world, enabling the audience to manipulate the shared experience. This level of interaction implicitly facilitates user interaction with the other virtual objects, which defines the narrative action.

The narrative of the play is implemented in a virtual environment development suite, Virtools (a VR authoring tool from Dassault Systems.) The virtual worlds (Figure 2) on different set are rendered by two PCs, and are automatically synchronized by the motion data. Hidden at a remote site, the sequestered actors can perceive live events on the display set by using networked camera and microphone feeds, and adapt their virtual counterparts’ behavior to react to the participants actions. This allows the audience members to take actions that influence the performance, and enables them to take an active role in the development of the improvised narrative. PERFORMANCE AND DESIGN RESEARCH

Figure 3 shows the performance installed at Canada’s Banff Centre. The design reveals a new concept of the audience in the role of co-producer and co-performer. The show also explores the relationship between the real and the virtual world where a mythological creature called the Trickster plays with the audience uneasiness with the virtual world. Audiences are presented with the virtual character, the Trickster, who initially dances around and creates his own world, but occasionally surprises the audience by truly interacting with them – creating a sense of uneasiness that this Trickster may be real in some way. As the Trickster plays in his magical forest, he is able to create and animate whimsical trees, mushrooms, and various plants. Gradually participants are made aware that their actions influence the Trickster’s behavior. For example, the Trickster can tease the visitor/player by duplicating his/her motions, or by responding to the visitor’s verbal or gestural commands to grow flowers by correspondingly moving to trigger the animation of growing flowers in the virtual world. The experience brings the audience and the character together at the intersection between the physical and the virtual world. Comparing to another virtual 1

This research investigates how people interact with playful technologies in virtual worlds and performing arts by leveraging interdisciplinary expertise. By transitioning the audience from their conventional passive role in a theatrical performance towards an actively collaborative role, the system realizes improvisation not only in performance but also in narrative creation. REFERENCES

1. M. Mateas and A. Stern. Facade: An experiment in building a fully-realized interactive drama. In Game developer’s Conference: Game Design Track, 2003. 2. C. Reeve. Presence in virtual theater. Presence: Teleoper. Virtual Environ., 9(2):209–213, 2000. 3. Q. Wu, P. Boulanger, M. Kazakevich, and R. Taylor. A real-time performance system for virtual theater. In Proceedings of the 2010 ACM workshop on Surreal media and virtual cloning, pages 3–8, 2010.

http://www.reintek.com/index.htm

2