Media and Embodiment

The “One-Person Choir”: a multidisciplinary approach to the development of an embodied human–Computer interface

by Pieter-Jan Maes

The “One-Person Choir” is a human–computer interface for singers that facilitates gestural control over a digital... more The “One-Person Choir” is a human–computer interface for singers that facilitates gestural control over a digital signal processing (DSP) module for harmonizing the singing voice in real time (see Figure 1). Harmonization adds extra pitch-shifted voices that are tonally related to the input voice. The interface captures global movements of the upper limbs by means of an integrated network of inertial sensors attached to the upper body of a singer. From these data, gestural cues are extracted and compared with a preconfigured gestural model that has been trained with empirical data. When the gestures of the singer match the preconfigured model, it is possible to control the harmonization of the singing input voice captured by a microphone. Thus, the interface allows a singer to naturally enhance the expressive qualities of his or her voice with the assistance of expressive gestures connected to an electronic environment. The One-Person Choir can be integrated in interactive multimedia installations that exploit the expressive power of gestures in combination with singing. As will be argued in this article, installations illustrate, and elaborate on, an ongoing shift in contemporary electronic and electroacoustic music: the move from interactive systems (or hyperinstruments) to composing interactions (Di Scipio 2003).

Is this my hand I see before me? The rubber hand illusion in reality, virtual reality, and mixed reality

by Wijnand IJsselsteijn

PRESENCE: Teleoperators and Virtual Environments 2006

The Body as a Medium: Reassessing the Role of Kinesthetic Awareness in Interactive Applications

by Aaron Levisohn

The popularity of Nintendo’s Wii is indicative of a fundamental
change in the relationship between humans and... more The popularity of Nintendo’s Wii is indicative of a fundamental
change in the relationship between humans and computers. After
decades of subservience to the keyboard and mouse, there is
now a reawakening of our kinesthetic and proprioceptive senses.
In this paper I am suggesting that the body be understood not
merely as the controller of multimedia systems, but as a
component within these systems and as a unique medium unto
itself. I will demonstrate an application of this idea by
describing a mixed reality system that was constructed to assist
with the exploration of proprioceptive and kinesthetic
awareness. The paper also includes a review of the informal
experiments that I conducted using this system to explore
possible kinesthetic and proprioceptive illusions.

Taro(t)ception: Eliciting Embodied, Interoceptive Awareness through Interactive Art

by Aaron Levisohn

Co-authored with Diane Gromala

This paper brings together multiple theories regarding the role of
the senses in the construction of embodied... more This paper brings together multiple theories regarding the role of
the senses in the construction of embodied experiences.
Embodiment, we suggest, is not a visual or auditory
phenomenon, but rather an ontological one, that is, one of being.
Employing accounts from cognitive science, existential
phenomenology, and interactive art, we argue that the inner
senses have a special role in the construction of these
ontological experiences.
We present an interactive artwork titled Taro(t)ception designed
to elicit an embodied aesthetic experience and heighten
awareness of inner states. As well as being an artwork,
Taro(t)ception is an exploration: the system provides a tool
through which we can explore proprioceptive illusions in order
to develop methods for transforming viewers' experiences of
their own bodies and their own movements.
Our approach attempts to bridge the gap between third-person
investigations, which rarely take in to account the quality of
experience, and first-person accounts, which are easily
dismissed as anecdotal.