Puppet UIinterface device

At present, a users' control of his/her persona in a 3D virtual environment such as "World of Warcraft" or "Second Life" is constrained to text input through a keyboard, excluding most forms of nonverbal and paraverbal communication. The "Puppet UI" is an ongoing exploration of alternative input methods, inspired by hand puppets, designed to enhance inter-personal communication and user agency in virtual environments.

Out proposed input methods aim to facilitate expressing bodily and vocal signals in CMC (computer mediated communication) and enhance the experience of social emotional communication in computer mediated environments. We have experimented with an input system which allow users to control their on-line avatar by directly manipulating a physical puppet.

While originating as a collaboration between Li Xin and myself, the "Puppet UI" found a home in Media Lab Helsinki's class on 3D user interfaces, taught by Markku Reunannen.

We utilized three different methods to develop the concept:

• background research - investigating similar projects, and applicable theory
• user studies - asking potential users to demonstrate their use of a real hand puppet, and translating the results to design ideas and requirements.
• expert consultation with a professional puppeteer.

We are developing the puppet UI as an iterative process, with multiple prototypes and rapid changes in the design concept

We have built a proof-of-concept prototype (as seen in the picture at the top), using a colour-coded glove in conjunction with a web camera to manipulate a 2D avatar. Our process, research and results are documented in the paper "The Puppet UI: Tools for nonverbal communication in virtual environments", forthcoming in Media Lab Working Papers 2008.