Interactive Media

Digital systems for live multimodal performance in Death and the Powers

by Peter Torpey

published in 'International Journal of Performance Arts and Digital Media'

The opera Death and the Powers by Tod Machover tells the story of a man, Simon Powers, who evades death by... more The opera Death and the Powers by Tod Machover tells the story of a man, Simon Powers, who evades death by transferring his essence into his environment as his corporeal body dies. To realize the effect of the theatrical environment coming alive as the main character, the Opera of the Future research group at the MIT Media Lab developed new technologies and control systems for interactive robotics, sound and visuals in live theatre. A core component of this work is the technique of Disembodied Performance, a method and associated technological infrastructure that translates the live performance of the offstage opera singer into multimodal representations onstage. The author was principally responsible for the control architecture and Disembodied Performance software implementation, as well as the design of the visual language used to represent Simon Powers. These digitally-enabled elements were created in order to support the story of the opera and facilitate the process of crafting and rehearsing the staged experience. This article reflects on the dialogue between the design of the technological systems in conjunction with the development of the story and scenography of the opera. Several design principles are presented for the role of new technologies in digital opera and music-driven performance contexts that arose during the course of this work. The discussed methods of cuing, authoring, organizing and collaborating suggest an approach for scoring the multimedia elements of digitally augmented stagecraft.

The Case for the Slow Internet

by Jennifer Proctor

Part of In Media Res's theme week on Interactive Documentaries.

nime05

by Michael Lyons

Goa Hippy Tribe: Theorising documentary content on a social network site

by Kate Nash

Published in Media International Australia no. 142 Feb 2012

Goa Hippy Tribe is the first Australian documentary to be produced for the social network site Facebook. In this... more Goa Hippy Tribe is the first Australian documentary to be produced for the social network site Facebook. In this article, I consider how documentary in a social network context might be theorised. While the concept of the database narrative is most often invoked to explain user interactivity in online documentary, social networks such as Facebook invite different forms of interaction and therefore raise distinct theoretical questions. In particular, Goa Hippy Tribe demonstrates the potential for the audience to engage creatively and communally with documentary

Cognitive Issues in Computer Music Programming

by Hiroki Nishino

In Proceedings of the International Conference on New Interfaces for Musical Expression 2011

Programming Languages are the oldest ‘new interface for music expression’ in computer music history. Both composers... more Programming Languages are the oldest ‘new interface for music expression’ in computer music history. Both composers and researchers in computer music still have considerable interests in computer music programming environments. However, while many researchers focus on such issues as efficiency, new paradigm, or new features in computer music programming, cognitive aspects of computer music programming has been rarely discussed. Such ‘cognitive issues’ are of importance when design or usability in computer music programming must be considered. By contextualizing computer music programming in the psychology of programming, it is made possible to borrow the technical terms and theoretical framework from the previous research in the field, which would be helpful to clarify the problems related to cognitive ergonomics and also beneficial to design a new programming environment with better usability in computer music.

A Shape-Free, Designable 6-DoF Marker Tracking Method for Camera-Based Interaction in Mobile Environment

by Hiroki Nishino

In Proceeding of ACM Multimedia 2010

We developed a novel marker tracking method with shape-free, designable markers, which can be visually meaningful to... more We developed a novel marker tracking method with shape-free, designable markers, which can be visually meaningful to users. The method can work fast enough to provide a real-time camera- based interaction even on low performance CPUs such as ones used in mobile Internet devices. Features such as visually communicative design and inexpensive computational cost are very desirable for users with mobile devices in the mobile/pervasive interaction environment.
The method utilizes the topological region adjacency to detect the marker candidates and then apply a simple method similar to geometric-hashing to determine the detected maker by voting to the hash tables. By such a combination of two different approaches, our method can distinguish those markers with the same topological structure and is also capable of 6-DoF pose estimation whereas most of the existing topology-based systems can not distinguish markers with the same topological structure and are incapable of 6-DoF pose estimation.

A Shape-Free, Designable 6-DoF Marker Tracking Method

by Hiroki Nishino

In Proceeding of ACM SIGGRAPH 2010

Markers are widely used for camera-based interaction. Yet, most of the marker tracking methods have considerable... more Markers are widely used for camera-based interaction. Yet, most of the marker tracking methods have considerable limitations in shapes and designs; they are not usually visually meaningful to the users.
Such an issue on visually communicative designs can be very important to provide visual cues in a mobile/pervasive environment where a user must first notice a marker and understand its meaning before initiating interaction, unlike in an immersive environment with a head-mounted-display that keeps on displaying some information on the detected markers.
We developed a novel tracking method with shape-free designable markers, based on topological region adjacency and geometric hashing. It can also distinguish the different markers with the same topological structure and provide 6- DoF pose estimation. Lack of such features has also been significant deficits of the existing topology-based methods.

A 6DoF Fiducial Tracking Method Based on Topological Region Adjacency and Angle Information for Tangible Interaction

by Hiroki Nishino

In Proceeding of Tangible, Embedded and Embodied Interaction 2010.

In this paper, we describe a new method for camera-based fiducial tracking. Our new method is based on the combination... more In this paper, we describe a new method for camera-based fiducial tracking. Our new method is based on the combination of topological region adjacency and angle information, where as related works by Johnston’s RAG target [7], Costanza’s D-Touch [3], and Kaltenbrunner’s reacTIVision [2] are based on the uniqueness of the topological region adjacency structure.
Such a combination of the topological region adjacency and angle information enables a wider unique ID range, while maintaining the merit of fast and robust fiducial tracking in topology-based approach. Our method makes it possible to obtain the 6 degrees-of-freedom (6DoF). Such problems of a narrow unique ID range and lack of 6DoF information have been the main deficits in most systems based on topological region adjacency approach, when compared to other fiducial tracking methods.

A Split-Marker Tracking Method based on Topological Region Adjacency & Vector Information for Interactive Card Games

by Hiroki Nishino

In Proceeding of ACM SIGGRAPH ASIA 2009

The extension of card games by interactive systems recently has been rapidly popular in gaming industry. Camera-based... more The extension of card games by interactive systems recently has been rapidly popular in gaming industry. Camera-based Tracking can play an important role to build such an interactive card games in low cost, without any special hardware. Split markers can be useful for such card games since the space between the markers can be placed a picture that is visually meaningful for players as in [2].
Among the criteria required for such a marker-tracking system for interactive card games, the robustness and the fast processing speed are important for better game experience; frequent false detection is a significant obstacle to identify and track the cards and slow processing speed is harmful for user interaction. We develop a novel split marker tracking method based on topological region adjacency and vector information. Our method can achieve fast processing speed and the robustness against false positive detection.

Topolo Surface: A 2D Fiducial Tracking System Based on Topological Region Adjacency and Angle Information

by Hiroki Nishino

IPSJ Journal Vol. 51 No. 2 240–249 (Feb. 2010)

In this paper, we describe Topolo Surface, a 2D fiducial tracking system we developed. Topolo Surface is a prototype... more In this paper, we describe Topolo Surface, a 2D fiducial tracking system we developed. Topolo Surface is a prototype system that implements a novel fidu- cial tracking method based on the combination of topological region adjacency and angle information. Existing systems based only on topological region adja- cency information, such as D-Touch and ReacTIVision, have several desirable features including fast processing speed and robustness against false positive detection. Yet, the method used in these systems also has several deficits. The unique ID range in existing topology-based methods is very narrow and the cost to generate the set of such unique fiducial markers can be computationally very expensive, especially when compared to existing matrix-based systems. Also, several useful techniques to improve robustness, such as CRC or hamming dis- tance, cannot be applied to existing topology-based systems. Our novel fiducial tracking method utilizes the combination of topological region adjacency and angle information. By using topological information together with geometri- cal information, our prototype system achieved much larger unique ID range at very cheap computational cost to generate its fiducial markers. This is achieved while maintaining the desirable features of fast processing speed and robust- ness against false positives in a topology-based method. Also, CRC or hamming distance can be applied to our method to improve the robustness, if necessary.

A 2D Fiducial Tracking Method Based on Topological Region Adjacency and Angle Information

by Hiroki Nishino

In Proceeding of New Interfaces for Musical Expression (NIME), 2009

We describe a new method for 2D fiducial tracking. We use region adjacency information together with angles between... more We describe a new method for 2D fiducial tracking. We use region adjacency information together with angles between regions to encode IDs inside fiducials, whereas previous research by Kaltenbrunner and Bencina utilize region adjacency tree. Our method supports a wide ID range and is fast enough to accommodate real-time video. It is also very robust against false positive detection.

An interactive approach to display large sets of association rules

by José Rouillard

Increasing reading and communication skills in children with autism through an interactive multimedia computer program

by Chris Chichester

Strangling philosophy

by Tawadeen Bankole

Interactivity and revisits to websites: a theoretical framework

by Nik Dholakia

Drama and Context in Real-Time Virtual Environments: Use of Pre-Scripted Events as a Part of an Interactive Spatial Mediation Framework

by Stanislav Roudavski

Cite as: Nitsche, Michael, Stanislav Roudavski, Maureen Thomas and François Penz (2003). 'Drama and Context in Real-Time Virtual Environments: Use of Pre-Scripted Events as a Part of an Interactive Spatial Mediation Framework', in Proceedings of 1st International Conference on Technologies for Interactive Digital Storytelling and Entertainment, ed. by Stefan Göbel et al. (Darmstadt: Fraunhofer IRB Verlag), pp. 296-310

We suggest that the dramatically engaging mediation of an experience of place should be built in as a fundamental... more We suggest that the dramatically engaging mediation of an experience of place should be built in as a fundamental capability of a compelling and meaningful virtual environment (VE). Our main objective is to develop flexible interactive techniques that supply VE’s with a coherent context and make the resulting ‘virtual place’ available to the user in a dramatically engaging way. To support the concept of narrative expressive space, we propose a three-layer multi-purpose spatial mediation framework that utilizes an interactive narrative structure to coordinate stylized dramatic camera work, lighting, effects and sound. We then describe the use of pre-scripted events as a layer in this framework and explain the inherent benefits and problems, using a single-user prototype environment as illustration. The work offers guidelines for the design of VE’s to all fields that combine narrativity and spatiality, such as interactive entertainment, education and architecture.

Internet-based interactive television: Case studies in experimenting with synchronous content interactivity

by CarrieLynn Reinhard

The most recent version (at Scribd) and first version (uploaded) of my research considering how television producers are utilizing Internet-based platforms for more interactive television experiences.

Studying the interpretive and physical aspects of interactivity: Revisiting interactivity as a situated interplay of structure and agencies

by CarrieLynn Reinhard

For a special issue of Communications on digital media revisited...

The concept of “interactivity” has... more For a special issue of Communications on digital media revisited...

The concept of “interactivity” has routinely been used to differentiate older analogue media and newer digital media. In this usage, interactivity has come to be defined as primarily a physical behavior from the person, as dictated by the media product, which has technological and/or content features that enable, promote, and require specific types and amounts of such activity. However, physical behaviors are only part of the processes involved in engaging with a media product. These also involve cognitive, affective and interpretive behaviors. Additionally, what are considered the most important behaviors may vary in any given media reception situation. This paper reports on a study that considered interactivity as involving interpretive and physical behaviors together. In interviews about people's engaging with new and old media products, the processes of interactivity were mapped for their interconnected components. The results help illustrate the complexity of the concept.

Experimental Media Horizons - Syncretic and Subversive Practices

by Joey Bargsten

Virtual Presentation: ICERI 2009

This paper formed part of the foundation for my book, Hybrid Forms and Syncretic Horizons (2011), available on Amazon... more This paper formed part of the foundation for my book, Hybrid Forms and Syncretic Horizons (2011), available on Amazon at [http://badmindtime.nfshost.com/amazon2].

Performative media--including video jamming, live and networked media collaboration, and mobile-device based performance--is reaching a critical mass in digital culture. In Experimental Media Horizons, we will examine a number of outlying practices where the materials of digital culture are beginning to coalesce, and place current activity into a broader historical and aesthetic context. Included is an overview of tech requirements and performance suggestions for developing your own course.

String Quartet (2010)

by Joey Bargsten

(draft only) Requires the use of mNOTATION™ online random music notation generation software (http://webprototypes.wordpress.com).

String Quartet (2010) is a score for traditional string quartet, plus digital audio, using the mNOTATION™ online... more String Quartet (2010) is a score for traditional string quartet, plus digital audio, using the mNOTATION™ online random music notation generating software.