User Interface

Controlling your game controls: interface and customization

by Sonam Adinolf

Co-authored with Selen Turkay

A game's interface is where players communicate with the game, so it has intrinsic importance to players. As player... more A game's interface is where players communicate with the game, so it has intrinsic importance to players. As player interactions in a game get more advanced, so does the complexity of the game interface. Massively Multiplayer Online game (MMO) interfaces are at the top of the complexity pyramid as they can display plentiful information such as character skills, stats, maps, chat windows. As the interface gets more complex, so does the merits of customizable interfaces. This paper reports results from a study which investigated the importance of interfaces and interface customization for MMO players using an online mixed method survey. Results validate that interface quality is important for players and interface customization is a desirable feature for player engagement and motivation to play MMOs. Further results are discussed in the paper.

Multichat: persistent, text-as-you-type messaging in a web browser for fluid multi-person interaction and collaboration.

by Jon Schull

Towards a recommender system based on price estimation models

by Claudia Becerra

Abstract of the thesis:
The most common recommendation approaches are based on historic information. In those... more Abstract of the thesis:
The most common recommendation approaches are based on historic information. In those approaches the recommendations consist on similar products or products that have been considered by similar users. This thesis proposes a new
recommendation approach based on price product estimations that recommends those products that present the best relation value-price (“the deals”) in comparison with the other products in the same class.
Furthermore, if the price estimation model used is a linear model that calculates the contribution of each product attribute to the price, the model can be considered as a compensatory multi-attribute decision model such as a simple additive weighting model, in which initially the weights are automatically assessed.
However and given the poor prediction performance obtained using linear models as price estimation models, a new aggregated regression model was proposed called 3-Functions Membership model (3FMM) that retains the interpretability of the initial linear models at the same time that classifies the products in three price subcategories.
Following a mixture of experts scheme, the 3FMM learns how to divide the input space in 3 subspaces or categories, and for each subspace a linear price estimation model is trained. The final estimation is the average of the three estimations weighted by the membership level of the instance to each subspace.
The main motivation behind the 3FFM is to determine the inferior and superior limit in which the price oscillates and to learn in which proportion a particular instance belongs to each one of the subspaces.
The results of the experiments using the 3FMM with typical machine learning datasets, showed a decreased error margin compared with the initial model margin, when the behavior of the initial model is weak and stable. As a case study, a 1381 laptops dataset with 69 attributes was gathered from several e-commerce sites. The application of the recommendation model to this dataset made evident the usefulness of the approach in those datasets in which is possible to train a hedonic price estimation model.

SmartSnap: addressing 3D pointing anisotropy in Virtual Reality CAD application

by Michele Fiorentino

Principles of human-computer interaction in fieldwork software. Study of the archaeologist as a user and its implications

by Patricia Martín-Rodilla

Fusion in Multimodal Interactive Systems: An HMM-Based Algorithm for User Induced Adaptation

by Beat Signer

Bruno Dumas, Beat Signer and Denis Lalanne, Proceedings of EICS 2012, 4th International Conference on Engineering Interactive Computing Systems, Copenhagen, Denmark, June 2012

Multimodal interfaces have shown to be ideal candidates for interactive systems that adapt to a user either... more Multimodal interfaces have shown to be ideal candidates for interactive systems that adapt to a user either automatically or based on user-defined rules. However, user-based adaptation demands for the corresponding advanced software architectures and algorithms. We present a novel multimodal fusion algorithm for the development of adaptive interactive systems which is based on hidden Markov models~(HMM). In order to select relevant modalities at the semantic level, the algorithm is linked to temporal relationship properties. The presented algorithm has been evaluated in three use cases from which we were able to identify the main challenges involved in developing adaptive multimodal interfaces.

Haptic Palpation for Medical Simulation in Virtual Environments

by Sebastian Ullrich

Ullrich S & Kuhlen T: “Haptic Palpation for Medical Simulation in Virtual Environments”, In IEEE Transactions on Visualization and Computer Graphics. 2012, April. 18(4):617-625. IEEE Press. Best Paper Award.

Palpation is a physical examination technique where objects, e.g., organs or body parts, are touched with fingers to... more Palpation is a physical examination technique where objects, e.g., organs or body parts, are touched with fingers to determine their size, shape, consistency and location. Many medical procedures utilize palpation as a supplementary interaction technique and it can be therefore considered as an essential basic method. However, palpation is mostly neglected in medical training simulators, with the exception of very specialized simulators that solely focus on palpation, e.g., for manual cancer detection. In this article we propose a novel approach to enable haptic palpation interaction for virtual reality–based medical simulators. The main contribution is an extensive user study conducted with a large group of medical experts. To provide a plausible simulation framework for this user study, we contribute a novel and detailed interaction algorithm for palpation with tissue dragging, which utilizes a multi–object force algorithm to support multiple layers of anatomy and a pulse force algorithm for simulation of an arterial pulse. Furthermore, we propose a modification for an off–the–shelf haptic device by adding a lightweight palpation pad to support a more realistic finger grip configuration for palpation tasks. The user study itself has been conducted on a medical training simulator prototype with a specific procedure from regional anesthesia, which strongly depends on palpation. The prototype utilizes a co–rotational finite element approach for soft tissue simulation and provides bimanual interaction by combining the aforementioned techniques with needle insertion for the other hand. The results of the user study suggest reasonable face validity of the simulator prototype and in particular validate medical plausibility of the proposed palpation interaction algorithm.

Navigation and Devices

by Monika Fleischmann

by John Bowers, Monika Fleischmann, Wolfgang Strauss, Jeffrey Shaw, Sten-Olof Hellström, Michael Hoch, Jaeae-Aro, Thomas Kulessa, Jasminko Novak, CID-81, KTH, Stockholm, Sweden May 1998

eRena Project, Deliverable 6.1 is concerned with developing new interfaces and new metaphors for more physical... more eRena Project, Deliverable 6.1 is concerned with developing new interfaces and new metaphors for more physical interaction with virtual environments, involving the entire body and its physical properties.

Spatio-Temporal Proximity as a Basis for Collaborative Filtering in Mobile Environments

by Beat Signer

Alexandre de Spindler, Moira C. Norrie, Michael Grossniklaus and Beat Signer, Proceedings of UMICS 2006, Workshop on Ubiquitous Mobile Information and Collaboration Systems, CAiSE 2006, 18th International Conference on Advanced Information Systems Engineering. Luxembourg, Grand Duchy of Luxembourg, June 2006

We propose a new approach to collaborative filtering in mobile tourist information systems based on spatio-temporal... more We propose a new approach to collaborative filtering in mobile tourist information systems based on spatio-temporal proximity in social contexts. Users store ratings and reviews of locations and events locally within their personal information system and then exchange these with other users present in the same social contexts using ad-hoc networking and opportunistic information sharing. We describe some preliminary investigations on the use of these methods in a mobile information system for visitors to a festival.

iGesture: A Java Framework for the Development and Deployment of Stroke-Based Online Gesture Recognition Algorithms

by Beat Signer

Beat Signer, Moira C. Norrie and Ueli Kurmann, Technical Report ETH Zurich, TR561, September 2007

Existing gesture recognition tools and frameworks tend to focus on specific settings, gesture sets or algorithms.... more Existing gesture recognition tools and frameworks tend to focus on specific settings, gesture sets or algorithms. Further, they are often designed to support the developers of either applications or algorithms, but not both. Our goal was to develop a general and extensible framework that provides an integrated platform for the design and evaluation of algorithms, as well as for their deployment to a wide audience. The presented iGesture framework supports the definition and evaluation of new gesture sets. Furthermore, our gesture recognition framework enables an easy integration of new forms of input devices. We present the iGesture framework, show how it has been used to support the development of two new gesture recognition algorithms—an extension of Rubine called E-Rubine and SiGrid—and finally provide an evaluation of these algorithms.

PaperProof: A Paper-Digital Proof-Editing System

by Beat Signer

Nadir Weibel, Beat Signer, Patrick Ponti and Moira C. Norrie, CoPADD 2007, 2nd International Workshop on Collaborating over Paper and Digital Documents, London, UK, November 2007

Recent approaches for linking paper and digital information or services tend to be based on a one-time publishing of... more Recent approaches for linking paper and digital information or services tend to be based on a one-time publishing of digital information where changes to the printed document become isolated from its digital source. Structural information which is available when authoring a digital document is lost during the printing process making it difficult to map interactions within the physical document to the corresponding elements of the digital document. We identify the necessary requirements for an integrated digital and paper-based document lifecycle and present our solution which supports a seamless transition between digital and physical document instances. PaperProof is presented as a paper-based proof-editing application that exploits our new approach for mapping pen-based interactions with paper documents to the corresponding operations in the digital document instance.

Towards Query by Sketch

by Beat Signer

Michael Springmann, Adriana Ispas, Heiko Schuldt, Moira C. Norrie and Beat Signer, Second DELOS Conference on Digital Libraries, Pisa, Italy, December 2007

Content-based retrieval has become a very popular and also powerful paradigm for searching in multimedia collections,... more Content-based retrieval has become a very popular and also powerful paradigm for searching in multimedia collections, especially in large collections of images. However, such queries require that one or even several reference images are available prior to the start of the search process. These reference images must be close to the final result so that the user can take them to express her information need. If such reference images are not available or if the information need is covered only by parts of the query object, the result usually does not meet the user’s expectation. Therefore, more flexible user interfaces are needed that allow users to sketch a query image by hand drawings and to dynamically select regions of interest from a given query image. In this paper, we present a novel approach to query by sketch where interactive paper and image similarity search are seamlessly combined. It is based on the iPaper/iServer system of ETH Zurich and the ISIS/OSIRIS content-based image retrieval system of the University of Basel. The paper presents the integrated system which has already very successfully been applied to the development of an interactive museum catalogue. Moreover, it reports on ongoing activities that aim at extending the system to support handwritten sketches, gestures and/or dynamic region selection to make the retrieval process more flexible and less dependent from existing query objects.

A Graphical UIDL Editor for Multimodal Interaction Design Based on SMUIML

by Beat Signer

Bruno Dumas, Beat Signer and Denis Lalanne, Proceedings of UIDL 2011, Workshop on Software Support for User Interface Description Language, Lisbon, Portugal, September 2011

We present the results of an investigation on software support for the SMUIML multimodal user interaction description... more We present the results of an investigation on software support for the SMUIML multimodal user interaction description language. In particular, we introduce a graphical UIDL editor for the creation of SMUIML scripts. The presented graphical editor is fully based on SMUIML for the representation of the underlying data as well as for the dialogue modelling. Due to the event-centered nature of SMUIML, the representation of the multimodal dialogue modelling in the graphical SMUIML dialogue editor has been realised via a state machine. The editor further offers a real-time graphical debugging tool. Compared to existing multimodal dialogue editors, the SMUIML graphical editor offers a dual graphical and textual editing as well as a number of operators for the temporal combination of modalities.

PowerPoint Multimedia Presentations in Computer Science Education: What do Users Need?

by Beat Signer

Elke I. Reuss, Beat Signer and Moira C. Norrie, Proceedings of USAB 2008, 4th Symposium on Usability & HCI for Education and Work, Graz, Austria, November 2008

PowerPoint is one of the most frequently used tools to present multimedia for educational purposes. Nevertheless,... more PowerPoint is one of the most frequently used tools to present multimedia for educational purposes. Nevertheless, little is known about the users' needs when using PowerPoint during lecturing. Our study focused on the presenter's needs in this context of use. We interviewed nine university lecturers from the computer science department by means of a questionnaire. The results show that users require features that are not yet realised with PowerPoint. For example, the control and use of other media should be adequately integrated. Or the navigation within the slide collection should be improved to better meet the users' needs. Based on our findings, we outline required system features and suggest solutions in form of a prototype.

Paper-Digital Meeting Support and Review

by Beat Signer

Adriana Ispas, Nan Li, Moira C. Norrie and Beat Signer, Proceedings of CollaborateCom 2010, 6th International Conference on Collaborative Computing, Chicago, USA, October 2010

Paper notes are still widely used during meetings for the capture and review of information created in meetings.... more Paper notes are still widely used during meetings for the capture and review of information created in meetings. However, personal notes are limited in terms of providing an overview of collaborative work practices and reflecting the evolution of data along successive meeting phases, especially taking into account actions performed on paper and digital media. We propose a solution for the review of meeting data captured along three dimensions of collaboration: paper-digital interaction, private and shared documents as well as pre- and in-meeting information. Based on a general data model, our system enhances the transition between succeeding meeting phases and improves the review of personal and collaborative cross-media meeting material.

Interactive Paper: Past, Present and Future

by Beat Signer

Beat Signer and Moira C. Norrie, Proceedings of PaperComp 2010, 1st International Workshop on Paper Computing, Copenhagen Denmark, September 2010

Over the last few years, there has been a significant increase in the number of researchers dealing with the... more Over the last few years, there has been a significant increase in the number of researchers dealing with the integration of paper and digital information or services. While recent technological developments enable new forms of paper-digital integration and interaction, some of the original research on interactive paper dates back almost twenty years. We give a brief overview of the most relevant past and current interactive paper developments. Then, based on our experience in developing a wide variety of interactive paper solutions over the last decade, as well as the results of other research groups, we outline future directions and challenges for the realisation of innovative interactive paper solutions. Further, we propose the definition of common data formats and interactive paper design patterns to ensure future cross-application and framework interoperability.

Midas: A Declarative Multi-Touch Interaction Framework

by Beat Signer

Christophe Scholiers, Lode Hoste, Beat Signer and Wolfgang De Meuter, Proceedings of TEI 2011, 5th International Conference on Tangible, Embedded and Embodied Interaction, Funchal, Portugal, January 2011

Over the past few years, multi-touch user interfaces emerged from research prototypes into mass market products. This... more Over the past few years, multi-touch user interfaces emerged from research prototypes into mass market products. This evolution has been mainly driven by innovative devices such as Apple's iPhone or Microsoft's Surface tabletop computer. Unfortunately, there seems to be a lack of software engineering abstractions in existing multi-touch development frameworks. Many multi-touch applications are based on hard-coded procedural low level event processing. This leads to proprietary solutions with a lack of gesture extensibility and cross-application reusability. We present Midas, a declarative model for the definition and detection of multi-touch gestures where gestures are expressed via logical rules over a set of input facts. We highlight how our rule-based language approach leads to improvements in gesture extensibility and reusability. Last but not least, we introduce JMidas, an instantiation of Midas for the Java programming language and describe how JMidas has been applied to implement a number of innovative multi-touch gestures.

Mudra: A Unified Multimodal Interaction Framework

by Beat Signer

Lode Hoste, Bruno Dumas and Beat Signer, Proceedings of ICMI 2011, 13th International Conference on Multimodal Interaction, Alicante, Spain, November 2011

In recent years, multimodal interfaces have gained momentum as an alternative to traditional WIMP interaction styles.... more In recent years, multimodal interfaces have gained momentum as an alternative to traditional WIMP interaction styles. Existing multimodal fusion engines and frameworks range from low-level data stream-oriented approaches to high-level semantic in\-fer\-ence-based solutions. However, there is a lack of multimodal interaction engines offering native fusion support across different levels of abstractions to fully exploit the power of multimodal interactions. We present Mudra, a unified multimodal interaction framework supporting the integrated processing of low-level data streams as well as high-level semantic inferences. Our solution is based on a central fact base in combination with a declarative rule-based language to derive new facts at different abstraction levels. Our innovative architecture for multimodal interaction encourages the use of software engineering principles such as modularisation and composition to support a growing set of input modalities as well as to enable the integration of existing or novel multimodal fusion engines.

Fundamental Concepts for Interactive Paper and Cross-Media Information Spaces

by Beat Signer

Beat Signer, Dissertation ETH No. 16218. Zurich, Switzerland, 2006.

Buy from Amazon.co.uk: http://www.amazon.co.uk/Fundamental-Concepts-Interactive-Cross-Media-Information/dp/3837027139

While there have been dramatic increases in the use of digital technologies for information storage, processing and... more While there have been dramatic increases in the use of digital technologies for information storage, processing and delivery over the last twenty years, the affordances of paper have ensured its retention as a key information medium. Despite predictions of the paperless office, paper is ever more present in our daily work as reflected by the continuously increasing worldwide paper consumption.

Many researchers have argued for the retention of paper as an information resource and its integration into cross-media environments as opposed to its replacement. This has resulted in a wide variety of projects and technological developments for digitally augmented paper documents over the past decade. However, the majority of the realised projects focus on technical advances in terms of hardware but pay less attention to the very fundamental information integration and cross-media information management issues.

Our information-centric approach for a tight integration of paper and digital information is based on extending an object-oriented database management system with functionality for cross-media information management. The resulting iServer platform introduces fundamental link concepts at an abstract level. The iServer’s core link management functionality is available across different multimedia resources. Only the media-specific portion of these general concepts, for example the specification of a link’s source anchor, has to be implemented in the form of a plug-in to support new resource types. This resource plug-in mechanism results in a flexible and extensible system where new types of digital as well as physical resources can easily be integrated and, more importantly, cross-linked to the growing set of supported multimedia resources. In addition to the associative linking of information, our solution allows for the integration of semantic metadata and supports multiple classification of information units. iServer can, not only link between various static information entities, but also link to active content and this has proven to be very effective in enabling more complex interaction design.

As part of the European project Paper++, under the Disappearing Computer Programme, an iServer plug-in for interactive paper has been implemented to fully integrate paper and digital media, thereby gaining the best of the physical and the digital worlds. It not only supports linking from physical paper to digital information, but also enables links from digital content to physical paper or even paper to paper links. This multi-mode user interface results in highly interactive systems where users can easily switch back and forth between paper and digital information. The definition of an abstract input device interface further provides flexibility for supporting emerging technologies for paper link definition in addition to the hardware solutions for paper link definition and activation that were developed within the Paper++ project.

We introduce different approaches for cross-media information authoring where information is either compiled by established publishers with an expertise in a specific domain or by individuals who produce their own cross-media information environments. Preauthored information can be combined with personally aggregated information. A distributed peer-to-peer version of the iServer platform supports collaborative authoring and the sharing of link knowledge within a community of users.

The associations between different types of resources as well as other application-specific information can be visualised on different output channels. Universal access to the iServer’s information space is granted using the eXtensible Information Management Architecture (XIMA), our publishing platform for multi-channel access.

Our fundamental concepts for interactive paper and cross-media information management have been designed independently of particular hardware solutions and modes of interaction which enables the iServer platform to easily adapt to both new technologies and applications. Finally, the information infrastructure that we have developed has great potential as an experimental platform for the investigation of emerging multimedia resources in general and interactive paper with its possible applications in particular.

PaperPoint: A Paper-Based Presentation and Interactive Paper Prototyping Tool

by Beat Signer

Beat Signer and Moira C. Norrie, Proceedings of TEI 2007, First International Conference on Tangible and Embedded Interaction, Baton Rouge, USA, February 2007

Recent developments in digital pen and paper solutions enable, not only the digital capture of handwriting, but also... more Recent developments in digital pen and paper solutions enable, not only the digital capture of handwriting, but also paper to be used as an interactive medium that links to digital information and services. We present a tool that builds on technologies for interactive paper to enable PowerPoint presentations to be controlled from printed slide handouts. Furthermore, slides can be easily annotated during presentations by simply drawing on the printed version of the slide. As well as discussing the advantages of such a paper-based interface and initial findings on its use, we describe how we were also able to exploit it to provide a general prototyping tool for interactive paper applications.