Interactive Arts

nime05

by Michael Lyons

The BrightHearts Project: A New Approach to the Management of Procedure-Related Paediatric Anxiety

by George Khut

Co-authored with Dr Angie Morrow, Staff Specialist, Kids Rehab, The Children's Hospital at Westmead, and Dr Melissa Yogui Watanabe

In this paper we propose a multimedia, biofeedback approach to managing pain and procedure-related anxiety experienced... more In this paper we propose a multimedia, biofeedback approach to managing pain and procedure-related anxiety experienced by children undergoing painful recurrent procedures as part of their treatment in primary care settings, based on research currently under way at the Children’s Hospital at Westmead, Kids Rehab Dept.

We survey existing approaches to the management of procedure-related pain and anxiety, including recent research utilising Virtual/Augmented Reality Distraction techniques, and then outline an approach that uses a biofeedback controlled interactive artwork as a focus for children to explore how they can regulate aspects of their psychophysiology (autonomic nervous system responses) through a combination of breath and
attentive focus.

Our research aims to assess the potential of small, portable biofeedback-based interactive artworks to mediate the perception and performance of the body in paediatric care: as experienced by children undergoing painful recurrent procedures.

Cognitive Issues in Computer Music Programming

by Hiroki Nishino

In Proceedings of the International Conference on New Interfaces for Musical Expression 2011

Programming Languages are the oldest ‘new interface for music expression’ in computer music history. Both composers... more Programming Languages are the oldest ‘new interface for music expression’ in computer music history. Both composers and researchers in computer music still have considerable interests in computer music programming environments. However, while many researchers focus on such issues as efficiency, new paradigm, or new features in computer music programming, cognitive aspects of computer music programming has been rarely discussed. Such ‘cognitive issues’ are of importance when design or usability in computer music programming must be considered. By contextualizing computer music programming in the psychology of programming, it is made possible to borrow the technical terms and theoretical framework from the previous research in the field, which would be helpful to clarify the problems related to cognitive ergonomics and also beneficial to design a new programming environment with better usability in computer music.

A Shape-Free, Designable 6-DoF Marker Tracking Method for Camera-Based Interaction in Mobile Environment

by Hiroki Nishino

In Proceeding of ACM Multimedia 2010

We developed a novel marker tracking method with shape-free, designable markers, which can be visually meaningful to... more We developed a novel marker tracking method with shape-free, designable markers, which can be visually meaningful to users. The method can work fast enough to provide a real-time camera- based interaction even on low performance CPUs such as ones used in mobile Internet devices. Features such as visually communicative design and inexpensive computational cost are very desirable for users with mobile devices in the mobile/pervasive interaction environment.
The method utilizes the topological region adjacency to detect the marker candidates and then apply a simple method similar to geometric-hashing to determine the detected maker by voting to the hash tables. By such a combination of two different approaches, our method can distinguish those markers with the same topological structure and is also capable of 6-DoF pose estimation whereas most of the existing topology-based systems can not distinguish markers with the same topological structure and are incapable of 6-DoF pose estimation.

A Shape-Free, Designable 6-DoF Marker Tracking Method

by Hiroki Nishino

In Proceeding of ACM SIGGRAPH 2010

Markers are widely used for camera-based interaction. Yet, most of the marker tracking methods have considerable... more Markers are widely used for camera-based interaction. Yet, most of the marker tracking methods have considerable limitations in shapes and designs; they are not usually visually meaningful to the users.
Such an issue on visually communicative designs can be very important to provide visual cues in a mobile/pervasive environment where a user must first notice a marker and understand its meaning before initiating interaction, unlike in an immersive environment with a head-mounted-display that keeps on displaying some information on the detected markers.
We developed a novel tracking method with shape-free designable markers, based on topological region adjacency and geometric hashing. It can also distinguish the different markers with the same topological structure and provide 6- DoF pose estimation. Lack of such features has also been significant deficits of the existing topology-based methods.

A 6DoF Fiducial Tracking Method Based on Topological Region Adjacency and Angle Information for Tangible Interaction

by Hiroki Nishino

In Proceeding of Tangible, Embedded and Embodied Interaction 2010.

In this paper, we describe a new method for camera-based fiducial tracking. Our new method is based on the combination... more In this paper, we describe a new method for camera-based fiducial tracking. Our new method is based on the combination of topological region adjacency and angle information, where as related works by Johnston’s RAG target [7], Costanza’s D-Touch [3], and Kaltenbrunner’s reacTIVision [2] are based on the uniqueness of the topological region adjacency structure.
Such a combination of the topological region adjacency and angle information enables a wider unique ID range, while maintaining the merit of fast and robust fiducial tracking in topology-based approach. Our method makes it possible to obtain the 6 degrees-of-freedom (6DoF). Such problems of a narrow unique ID range and lack of 6DoF information have been the main deficits in most systems based on topological region adjacency approach, when compared to other fiducial tracking methods.

A Split-Marker Tracking Method based on Topological Region Adjacency & Vector Information for Interactive Card Games

by Hiroki Nishino

In Proceeding of ACM SIGGRAPH ASIA 2009

The extension of card games by interactive systems recently has been rapidly popular in gaming industry. Camera-based... more The extension of card games by interactive systems recently has been rapidly popular in gaming industry. Camera-based Tracking can play an important role to build such an interactive card games in low cost, without any special hardware. Split markers can be useful for such card games since the space between the markers can be placed a picture that is visually meaningful for players as in [2].
Among the criteria required for such a marker-tracking system for interactive card games, the robustness and the fast processing speed are important for better game experience; frequent false detection is a significant obstacle to identify and track the cards and slow processing speed is harmful for user interaction. We develop a novel split marker tracking method based on topological region adjacency and vector information. Our method can achieve fast processing speed and the robustness against false positive detection.

Topolo Surface: A 2D Fiducial Tracking System Based on Topological Region Adjacency and Angle Information

by Hiroki Nishino

IPSJ Journal Vol. 51 No. 2 240–249 (Feb. 2010)

In this paper, we describe Topolo Surface, a 2D fiducial tracking system we developed. Topolo Surface is a prototype... more In this paper, we describe Topolo Surface, a 2D fiducial tracking system we developed. Topolo Surface is a prototype system that implements a novel fidu- cial tracking method based on the combination of topological region adjacency and angle information. Existing systems based only on topological region adja- cency information, such as D-Touch and ReacTIVision, have several desirable features including fast processing speed and robustness against false positive detection. Yet, the method used in these systems also has several deficits. The unique ID range in existing topology-based methods is very narrow and the cost to generate the set of such unique fiducial markers can be computationally very expensive, especially when compared to existing matrix-based systems. Also, several useful techniques to improve robustness, such as CRC or hamming dis- tance, cannot be applied to existing topology-based systems. Our novel fiducial tracking method utilizes the combination of topological region adjacency and angle information. By using topological information together with geometri- cal information, our prototype system achieved much larger unique ID range at very cheap computational cost to generate its fiducial markers. This is achieved while maintaining the desirable features of fast processing speed and robust- ness against false positives in a topology-based method. Also, CRC or hamming distance can be applied to our method to improve the robustness, if necessary.

A 2D Fiducial Tracking Method Based on Topological Region Adjacency and Angle Information

by Hiroki Nishino

In Proceeding of New Interfaces for Musical Expression (NIME), 2009

We describe a new method for 2D fiducial tracking. We use region adjacency information together with angles between... more We describe a new method for 2D fiducial tracking. We use region adjacency information together with angles between regions to encode IDs inside fiducials, whereas previous research by Kaltenbrunner and Bencina utilize region adjacency tree. Our method supports a wide ID range and is fast enough to accommodate real-time video. It is also very robust against false positive detection.

Experimental Media Horizons - Syncretic and Subversive Practices

by Joey Bargsten

Virtual Presentation: ICERI 2009

This paper formed part of the foundation for my book, Hybrid Forms and Syncretic Horizons (2011), available on Amazon... more This paper formed part of the foundation for my book, Hybrid Forms and Syncretic Horizons (2011), available on Amazon at [http://badmindtime.nfshost.com/amazon2].

Performative media--including video jamming, live and networked media collaboration, and mobile-device based performance--is reaching a critical mass in digital culture. In Experimental Media Horizons, we will examine a number of outlying practices where the materials of digital culture are beginning to coalesce, and place current activity into a broader historical and aesthetic context. Included is an overview of tech requirements and performance suggestions for developing your own course.

String Quartet (2010)

by Joey Bargsten

(draft only) Requires the use of mNOTATION™ online random music notation generation software (http://webprototypes.wordpress.com).

String Quartet (2010) is a score for traditional string quartet, plus digital audio, using the mNOTATION™ online... more String Quartet (2010) is a score for traditional string quartet, plus digital audio, using the mNOTATION™ online random music notation generating software.

Wakeful software and wakeful musical instruments: a theoretical approach to the implementation

by Fabio Paolizzo

to appear in: ISEA 2011 - conference proceedings

A new category of content-related software is here proposed. ‘Wakeful’ software is one that senses and responds to an... more A new category of content-related software is here proposed. ‘Wakeful’ software is one that senses and responds to an environment with which it may consistently interact meaningfully. The term ‘wakeful’ is here meant as ‘sign-bearing,’ rather than potentially being capable of such. Specifically, the study was carried out in relation to music and interactive music systems.

SINOPSE | PAISAGENS NEUROLÓGICAS

by Isabel Maria Pereira Dos Santos

Towards a theoretical framework for interactive digital narrative

by Hartmut Koenitz

The emerging artistic practice of interactive narrative in digital media marks a profound departure from traditional... more The emerging artistic practice of interactive narrative in digital media marks a profound departure from traditional narrative. The application of traditional narrative theory for interactive narrative is problematic, since the affordances of digital media challenge many underlying assumptions of theories related to non-digital media. This paper proposes a theoretical framework for interactive storytelling, which addresses these concerns by foregrounding system (the digital artifact) and process (the user interacting with the system) over the product-centered view of legacy media. On this basis, protostory, narrative design, and narrative vectors are proposed as new terms to more adequately describe the structure of narrative in interactive digital storytelling. This move is also relevant for practical design given the influence theoretical concepts have on concrete implementations.

Reframing interactive digital narrative: Toward an inclusive open-ended iterative process for research and practice

by Hartmut Koenitz

In more than two decades of research and practical experiments in interactive digital narrative (IDN), much insight... more In more than two decades of research and practical experiments in interactive digital narrative (IDN), much insight about the relationship of narrative and digital media has been gained and many successful experiments have been undertaken, as a survey of the field illustrates. However, current approaches also limit the scope of experimentation and constrain theory in interactive narrative forms original to digital media.

After reviewing the "interactivisation" of legacy theory (neo-Aristotelian poetics for interactive drama, poststructuralism for hyperfiction, 20th century narratology for interactive fiction and as a general theory for IDN), the thesis introduces a theoretical framework that changes the focus from the product-centered view of legacy media towards system and the process of instantiation. The terms protostory describing the overall space of potential narratives in an IDN system, narrative design for the concrete assemblage of elements and narrative vectors as substructures that enable authorial control are introduced to supersede legacy terms like story and plot.

On the practical side, the thesis identifies limitations of existing approaches (e.g. legacy metaphors like the timeline, and authoring tools that support only particular traditions) To overcome these limitations a software toolset built on the principles of robustness, modularity, and extensibility is introduced and some early results are evaluated. Finally, the thesis proposes an inclusive, open-ended iterative process as a structure for future IDN research in which practical implementations and research coexist in a tightly coupled mutual relationship that allows changes on one side to be integrated on the other.

VIVO (Video Interactive VST Orchestra) and the aesthetics of interaction

by Fabio Paolizzo

Published in SSSP 2010 - conference proceedings.

The research is drawn across the fields of musicology, composition and software development with the aim of achieving... more The research is drawn across the fields of musicology, composition and software development with the aim of achieving a collective intelligence and self-awareness through self-reflection in interactive music. The present study recognises that for a collective self-awareness to occur through self-reflection in music, specific structures of interrelation have to be adopted, which may enhance the human agent’s awareness of the own self as related to the machine. These structures are then implemented in a piece of software. Specific art projects are designed along the investigation to finally test/improve the framework through art practice. The art works spanned the disciplines of music, dance, theatre/performance, film-making, net-art, sport/music interactive public art. The implementations include: a module for adaptive video tracking that is derived from the feedback loop of action/perception (Vaggione 2001); an adaptive graphic score which is designed upon a previous multi-modal comparative analysis (Impett 2001); a dynamic host for audio software, where the concept of open content is merged within the dynamic orchestration model (Paine 2004). The piece of software, which provides outcomes that are informed by these structures, is VIVO. This software musical instrument is able to generate an adaptive musical answer to reflect the agents’ behaviour by controlling external audio-plug-ins (VST, DirectX, AU). The paper illustrates its main features, the theories behind the implementations and the partial evidence that was gathered from tests, which are still in progress, within specific art projects.

VIVO (Video Interactive VST Orchestra): an interactive and adaptive musical instrument for self-reflection in music

by Fabio Paolizzo

published in 'The International Journal of the Arts in Society'

The aim of the research is to create a software musical instrument that allows for an aware exploration of the self,... more The aim of the research is to create a software musical instrument that allows for an aware exploration of the self, through self-refection in music. The investigation is interdisciplinary and operates across the fields of musicology, software development and composition. The study focuses on specific logics of compatibility between human agents and technology. These are then implemented, at the design stage, in a software tool. Finally, the piece of software is tested within art projects to validate the implementations or to correct the theoretical assumptions and, therefore, to further improve the framework. This software musical instrument is VIVO (Video Interactive VST Orchestra). This paper illustrates the main features of VIVO, the theories behind the implementations and the evidence that was gathered from tests within two specific art projects.

[PAISAGENS NEUROLÓGICAS] | [PAESAGGI NEUROLOGICI]

by Isabel Maria Pereira Dos Santos

[WASTE OF LOVE AND OF THE HUMAN SPECIES]

by Isabel Maria Pereira Dos Santos

VIVO (Video Interactive VST Orchestra) and the aesthetics of interaction

by Fabio Paolizzo

Published in SSSP 2010 - conference proceedings.

The research is drawn across the fields of musicology, composition and software development with the aim of achieving... more The research is drawn across the fields of musicology, composition and software development with the aim of achieving a collective intelligence and self-awareness through self-reflection in interactive music. The present study recognises that for a collective self-awareness to occur through self-reflection in music, specific structures of interrelation have to be adopted, which may enhance the human agent’s awareness of the own self as related to the machine. These structures are then implemented in a piece of software. Specific art projects are designed along the investigation to finally test/improve the framework through art practice. The art works spanned the disciplines of music, dance, theatre/performance, film-making, net-art, sport/music interactive public art. The implementations include: a module for adaptive video tracking that is derived from the feedback loop of action/perception (Vaggione 2001); an adaptive graphic score which is designed upon a previous multi-modal comparative analysis (Impett 2001); a dynamic host for audio software, where the concept of open content is merged within the dynamic orchestration model (Paine 2004). The piece of software, which provides outcomes that are informed by these structures, is VIVO. This software musical instrument is able to generate an adaptive musical answer to reflect the agents’ behaviour by controlling external audio-plug-ins (VST, DirectX, AU). The paper illustrates its main features, the theories behind the implementations and the partial evidence that was gathered from tests, which are still in progress, within specific art projects.