Ubiquitous Computing

The Intelligent Classroom : Beyond Four Walls

by Daniyal Al Ghazzawi

Co-authored with J. Dooley, V. Callaghan, H. Hagras, M. Gardner, M. Ghanbari

It has become standard practice to use the traditional information technologies such as web and email in education.... more It has become standard practice to use the traditional information technologies such as web and email in education. But as the University campus becomes increasingly deployed with interconnected computing technology, we begin to ask how the resulting “iCampus” resource can be used in new and novel ways. This paper builds upon our previous work in the area of intelligent environments and goes beyond the typical use of computing technology within education. We report on our current research that aims to apply mixed reality and ubiquitous computing paradigms to enrich the teaching and learning experience and describe our deployments and innovations across the University of Essex campus.

Plastic interfaces for ubiquitous learning

by José Rouillard

This chapter presents research around pervasive and ubiquitous computing, particularly
oriented in the field of... more This chapter presents research around pervasive and ubiquitous computing, particularly
oriented in the field of human learning. We are studying several solutions to deliver content
over a heterogeneous networks and devices. Converting and transmitting documents across
electronic networks is not sufficient. We have to deal with contents and containers
simultaneously. Related work in interface adaptation and plasticity (the capacity of a user
interface to withstand variations of both the system physical characteristics and the
environment while preserving usability) is presented and some examples of context-aware
adaptation are exposed. We present an adaptive pervasive learning environment, based on
contextual QR Codes, where information is presented to learner at the appropriate time and
place, and according to a particular task. This learning environment is called PerZoovasive,
where learning activities take place in a zoo and are meant to enhance classroom activities.

Only Touching the Surface: Creating Affinities between Digital Content and Paper

by Beat Signer

Paul Luff, Christian Heath, Moira C. Norrie, Beat Signer and Peter Herdman, Proceedings of CSCW 2004, Conference on Computer Supported Cooperative Work. Chicago, USA, November 2004

Despite the wide-ranging recognition that paper remains a pervasive resource for human conduct and collaboration,... more Despite the wide-ranging recognition that paper remains a pervasive resource for human conduct and collaboration, there has been uncertain progress in developing technologies to bridge the paper-digital divide. In this essay we discuss the design of a technology that interweaves developments in new materials, electronics and software, and seeks to provide a cheap and accessible solution to creating new affinities between digital content, in whatever form, and ordinary paper. The technology and its design draws from a broad range of field studies, including research in classrooms and museums. These delineate the requirements and considerations that inform solutions to enhancing paper whilst preserving its integrity. The paper also discusses a naturalistic experiment, an evaluation in a museum, where we assessed the technology and the solution. We also chart the progressive development of this solution and the ways in which seemingly simple actions and issues became reconstituted as highly complex technical and analytic problems.

SpeeG: A Multimodal Speech- and Gesture-based Text Input Solution

by Beat Signer

Lode Hoste, Bruno Dumas and Beat Signer, AVI 2012, International Working Conference on Advanced Visual Interfaces, Capri Island, Italy, May 2012

We present SpeeG, a multimodal speech- and body gesture-based text input system targeting media centres, set-top boxes... more We present SpeeG, a multimodal speech- and body gesture-based text input system targeting media centres, set-top boxes and game consoles. Our controller-free zoomable user interface combines speech input with a gesture-based real-time correction of the recognised voice input. While the open source CMU Sphinx voice recogniser transforms speech input into written text, Microsoft's Kinect sensor is used for the hand gesture tracking. A modified version of the zoomable Dasher interface combines the input from Sphinx and the Kinect sensor. In contrast to existing speech error correction solutions with a clear distinction between a detection and correction phase, our innovative SpeeG text input system enables continuous real-time error correction. An evaluation of the SpeeG prototype has revealed that low error rates for a text input speed of about six words per minute can be achieved after a minimal learning phase. Moreover, in a user study SpeeG has been perceived as the fastest of all evaluated user interfaces and therefore represents a promising candidate for future controller-free text input.

Mudra: A Unified Multimodal Interaction Framework

by Beat Signer

Lode Hoste, Bruno Dumas and Beat Signer, Proceedings of ICMI 2011, 13th International Conference on Multimodal Interaction, Alicante, Spain, November 2011

In recent years, multimodal interfaces have gained momentum as an alternative to traditional WIMP interaction styles.... more In recent years, multimodal interfaces have gained momentum as an alternative to traditional WIMP interaction styles. Existing multimodal fusion engines and frameworks range from low-level data stream-oriented approaches to high-level semantic in\-fer\-ence-based solutions. However, there is a lack of multimodal interaction engines offering native fusion support across different levels of abstractions to fully exploit the power of multimodal interactions. We present Mudra, a unified multimodal interaction framework supporting the integrated processing of low-level data streams as well as high-level semantic inferences. Our solution is based on a central fact base in combination with a declarative rule-based language to derive new facts at different abstraction levels. Our innovative architecture for multimodal interaction encourages the use of software engineering principles such as modularisation and composition to support a growing set of input modalities as well as to enable the integration of existing or novel multimodal fusion engines.

Midas: A Declarative Multi-Touch Interaction Framework

by Beat Signer

Christophe Scholiers, Lode Hoste, Beat Signer and Wolfgang De Meuter, Proceedings of TEI 2011, 5th International Conference on Tangible, Embedded and Embodied Interaction, Funchal, Portugal, January 2011

Over the past few years, multi-touch user interfaces emerged from research prototypes into mass market products. This... more Over the past few years, multi-touch user interfaces emerged from research prototypes into mass market products. This evolution has been mainly driven by innovative devices such as Apple's iPhone or Microsoft's Surface tabletop computer. Unfortunately, there seems to be a lack of software engineering abstractions in existing multi-touch development frameworks. Many multi-touch applications are based on hard-coded procedural low level event processing. This leads to proprietary solutions with a lack of gesture extensibility and cross-application reusability. We present Midas, a declarative model for the definition and detection of multi-touch gestures where gestures are expressed via logical rules over a set of input facts. We highlight how our rule-based language approach leads to improvements in gesture extensibility and reusability. Last but not least, we introduce JMidas, an instantiation of Midas for the Java programming language and describe how JMidas has been applied to implement a number of innovative multi-touch gestures.

Ubiquitous Network in Tourism / Redes Ubicuas en Turismo

by Pablo Vanevic

draft only

This paper seeks to bring a critical space for discussion, the term ubiquitous network, its emphasis on the territory,... more This paper seeks to bring a critical space for discussion, the term ubiquitous network, its emphasis on the territory, manifested power, current and potential perception and its application to tourism.

El presente trabajo busca traer a un espacio crítico de debate, el término de redes ubicuas; su énfasis en el territorio, el poder manifiesto, su percepción actual y potencial y su aplicación al ámbito turístico, ya desde la óptica misma de la actividad como desde la academia.

AnswerTree – a collaborative mobile location-based educational game

by Elizabeth FitzGerald (née Brown)

Full citation:
Moore, A., Goulding, J., Brown, E. and J. Swan (2009) AnswerTree – a collaborative mobile location-based educational game. Proceedings of the mLearn 2009 Conference, Orlando, Florida, 26-30 Oct 2009, pp 199-202.

In this paper we present AnswerTree, a collaborative mobile location-based educational game designed to teach 8-12... more In this paper we present AnswerTree, a collaborative mobile location-based educational game designed to teach 8-12 year olds about trees and wildlife within the University of Nottingham campus. The activity is designed around collecting virtual cards (similar in nature to the popular Top Trumps games) containing graphics and information about notable trees. Each player begins by collecting one card from a game location, but then he or she can only collect further cards by answering questions – whose solutions are obtainable through sharing knowledge with other cardholders. This ostensibly allows each player to become a subject expert at the start of the game, encouraging collaborative interaction for the game to be successfully completed. In this initial paper we will outline the structure and background of this location based game. AnswerTree has been authored within the Hyperplace framework, and is a first implementation of a wider process to develop a flexible, multi-purpose platform for both individual and group location-based mobile learning.

Augmenting the field experience: A student-led comparison of techniques and technologies.

by Elizabeth FitzGerald (née Brown)

Full citation:
Priestnall, G., Brown, E., Sharples, M. and G. Polmear (2010) Augmenting the field experience: A student-led comparison of techniques and technologies. In: Brown, E. (ed) Education in the Wild: A report from the STELLAR Alpine Rendez-Vous workshop series. University of Nottingham: Learning Sciences Research Institute (LSRI). ISBN 9780853582649: pp 43-46.

In this study we report on our experiences of creating and running a student fieldtrip exercise which allowed students... more In this study we report on our experiences of creating and running a student fieldtrip exercise which allowed students to compare a range of approaches to the design of technologies for augmenting landscape scenes. The main study site is around Keswick in the English Lake District, Cumbria, UK, an attractive upland environment popular with tourists and walkers. The aim of the exercise for the students was to assess the effectiveness of various forms of geographic information in augmenting real landscape scenes, as mediated through a range of techniques and technologies. These techniques were: computer-generated acetate overlays showing annotated wireframe views from certain key points; a custom-designed application running on a PDA; a mediascape running on the mScape software on a GPS-enabled mobile phone; Google Earth on a tablet PC; and a head-mounted in-field Virtual Reality system. Each group of students had all five techniques available to them, and were tasked with comparing them in the context of creating a visitor guide to the area centred on the field centre. Here we summarise their findings and reflect upon some of the broader research questions emerging from the project.

Encouraging museum visitor engagement using spontaneous talk-in-interaction audio guides

by Elizabeth FitzGerald (née Brown)

Full citation:
Elliston, B. and E. FitzGerald (2012) Encouraging museum visitor engagement using spontaneous talk-in-interaction audio guides. Long paper accepted to the 4th International Conference on Computer Supported Education (CSEDU 2012), 16-18 April, Porto, Portugal.

We describe the building and testing of a museum audio tour with content recorded as spontaneous interactive dialogue... more We describe the building and testing of a museum audio tour with content recorded as spontaneous interactive dialogue between two curators as they walked around an art gallery. The aim was to produce a guide which would increase the amount of topically relevant talk shared by people visiting a museum in groups of two or more. Conversation analysis is used to show how a pair of visitors engaged more with the content of the guide than they would have with audio produced as traditional scripted monologue. Examples of a variety of engagement types are detailed and a supporting rationale drawing on Goffman's theory of 'footing' is discussed. The approach potentially offers a low cost way for organisations involved in informal learning to produce flexible in-house audio content for mobile and e-learning, which improves visitor engagement both with the content and with one another, and leads to a more enjoyable visitor/learner experience than traditional forms of audio.

Creating user-generated content for location-based learning: an authoring framework

by Elizabeth FitzGerald (née Brown)

FitzGerald, E. (2012) Creating user-generated content for location-based learning: an authoring framework. Journal of Computer Assisted Learning 28 (3) pp 195-207. doi: 10.1111/j.1365-2729.2012.00481.x

NB - the PDF here is a pre-print copy. For an official 'journal' version please email me and I'll send you the PDF.

Two recent emerging trends are that of Web 2.0, where users actively create content and publish it on the Web and also... more Two recent emerging trends are that of Web 2.0, where users actively create content and publish it on the Web and also location awareness, where a digital device utilises a person’s physical location as the context to provide specific services and/or information. This paper examines how these two phenomena can be brought together, so that user-generated content on mobile devices are used to provide informal learning opportunities relevant to a person’s location. However, the generative process of such media does not always have much guidance on how or what to create, so the quality of such information can be highly variable. To overcome this, a framework has been designed to guide the authoring of user-generated content so that it can be used for informal learning about one’s immediate surroundings (particularly in an outdoor setting), combining pedagogical aspects with those from human-computer interaction and environmental aesthetics. The framework consists of six dimensions that include aspects such as curriculum area (e.g. science; art); type of communication; use of language/media related to the landscape; knowledge level of content; contextual aspects and types of interaction. In order to test the framework before it could be used to scaffold new content, it was first used to analyse and evaluate over 200 items of existing user-generated content, to investigate the appropriateness of the proposed dimensions and the items contained therein or if any were missing. This paper presents the findings of this initial testing phase, together with a discussion of how the framework can be improved, in order to help scaffold the creation of new user-generated content in the future.

A student-led comparison of techniques for augmenting the field experience

by Elizabeth FitzGerald (née Brown)

Full citation:
Priestnall, G., Brown, E. and M. Sharples (2009) A student-led comparison of techniques for augmenting the field experience. Proceedings of the mLearn 2009 Conference, Orlando, Florida, 26-30 Oct 2009, pp 195-198.

We report a study in which 30 university geography students compared five techniques to enhance the experience of... more We report a study in which 30 university geography students compared five techniques to enhance the experience of visiting outdoor locations. The techniques were: a pre-prepared acetate overlay of the visual scene; a custom-designed visitor guide running on a PDA; the mScape location-based software running on a GPS-enabled mobile phone; Google Earth on a tablet PC; and a head-mounted virtual reality display. The students were given the assignment as part of their assessed coursework for a field trip to the UK Lake District, where they had to evaluate the techniques and propose improvements or future designs to enable tourists or students on field trips to gain an enhanced understanding of their surroundings. The paper describes these techniques, reports the process and results of the student assignment, and concludes with a discussion of some broader issues emerging from the project.

A participatory design agenda for ubiquitous computing and multimodal interaction: a case study of dental practice

by Margot Brereton

Through the augmenting-glass: an analysis on Augmented Reality and its remediation with mobile technology within the city environment

by Laura Lotti

Dissertation - MA Digital Media: Technology and Cultural Form

Together with ubiquitous computing and wearable technologies, augmented reality (AR) belongs to the triad of... more Together with ubiquitous computing and wearable technologies, augmented reality (AR) belongs to the triad of intelligent technologies that promise to simplify individuals’ lives thanks to their capability to allow users to access information anywhere and at any time. AR is a remarkably visual technology that aims at enhancing one’s experience of reality by allowing the user to visualize information in virtual format superimposed upon the physical space.
The aim of this work is to give an account of AR that reads behind the rhetoric of this new technology, that speaks a language on enhancement, and incremental rather than epochal changes. Augmentation in this sense has important implications for the way we perceive reality, but also for our perception of the environment itself, especially in the context of urban space.
Moreover, given the remediation of AR with mobile communication technology, the individual is supposed to interact with the surrounding through the screen of a mobile device, which brings to the fore issues related to the perception of the self and representationalism. As Donna Haraway puts it, specific ways of seeing correspond to specific ways of living. Therefore the question of how we see, linked to the development of AR, entails important reconfigurations in the way we perceive reality and ourselves immersed in it.
My aim will be to demonstrate that there is nothing new about AR, a part from a superfluous focus on the visual element that appears on the surface of a screen. If we learn how to look beyond the surface - with our embodied eyes, and embedded in the materiality that surrounds us – we may have a true augmented experience of reality, with or without fancy technological gadgets.

Through the augmenting-glass: an analysis on Augmented Reality and its remediation with mobile technology within the city environment

by Laura Lotti

Dissertation - MA Digital Media: Technology and Cultural Form

Together with ubiquitous computing and wearable technologies, augmented reality (AR) belongs to the triad of... more Together with ubiquitous computing and wearable technologies, augmented reality (AR) belongs to the triad of intelligent technologies that promise to simplify individuals’ lives thanks to their capability to allow users to access information anywhere and at any time. AR is a remarkably visual technology that aims at enhancing one’s experience of reality by allowing the user to visualize information in virtual format superimposed upon the physical space.
The aim of this work is to give an account of AR that reads behind the rhetoric of this new technology, that speaks a language on enhancement, and incremental rather than epochal changes. Augmentation in this sense has important implications for the way we perceive reality, but also for our perception of the environment itself, especially in the context of urban space.
Moreover, given the remediation of AR with mobile communication technology, the individual is supposed to interact with the surrounding through the screen of a mobile device, which brings to the fore issues related to the perception of the self and representationalism. As Donna Haraway puts it, specific ways of seeing correspond to specific ways of living. Therefore the question of how we see, linked to the development of AR, entails important reconfigurations in the way we perceive reality and ourselves immersed in it.
My aim will be to demonstrate that there is nothing new about AR, a part from a superfluous focus on the visual element that appears on the surface of a screen. If we learn how to look beyond the surface - with our embodied eyes, and embedded in the materiality that surrounds us – we may have a true augmented experience of reality, with or without fancy technological gadgets.

Design from the Everyday: Continuously evolving, embedded exploratory prototypes

by Margot Brereton

Published and Presented at the Designing Interavtive Systems Conference in Aarhus, Denmark 2010

ACM conference. 22% acceptance rate in the long paper track.

One of the major challenges in the design of social
technologies is the evaluation of their qualities of use... more One of the major challenges in the design of social
technologies is the evaluation of their qualities of use and
how they are appropriated over time. While the field of
HCI abounds in short-term exploratory design and studies
of use, relatively little attention has focused on the
continuous development of prototypes longitudinally and
studies of their emergent use. We ground the exploration
and analysis of use in the everyday world, embracing
contingency and open-ended use, through the use of a
continuously-available exploratory prototype. Through
examining use longitudinally, clearer insight can be gained
of realistic, non-novelty usage and appropriation into
everyday use.

This paper sketches out a framework for design that puts a
premium on immediate use and evolving the design in
response to use and user feedback. While such design
practices with continuously developing systems are common
in the design of social technologies, they are little
documented. We describe our approach and reflect upon its
key characteristics, based on our experiences from two case
studies. We also present five major patterns of long-term
usage which we found useful for design.

New challenges for design participation in the era of ubiquitous computing

by Margot Brereton

An application framework for open application development and distribution in pervasive display networks

by Constantin Taivan

Experiments in ubiquitous computing for communities of practice using learning resources

by José Rouillard

The Pervasive Fridge

by José Rouillard