Affective Computing

An effective application for tackling patient depression

by Marc Fabri

Fabri, M., Peter, C., Girard, S., Agius, H. (2008) An effective application for tackling patient depression, in Christian Peter, Russell Beale, Elizabeth Crane, Lesley Axelrod, Gerred Blyth (Eds.): Emotion in HCI: Joint Proceedings of the 2005, 2006, and 2007 Intl. Workshops. Fraunhofer IRB Verlag, Stuttgart, ISBN 978-3-8167-7540-9

This paper is a result of a working group at the 3rd Workshop on Emotion in HCI. The aim of this working group was to... more This paper is a result of a working group at the 3rd Workshop on Emotion in HCI. The aim of this working group was to sketch an affective system for the support of depressed people living mainly on their own. In this paper we present some results of this work, focusing on how acquired emotion information could be processed and provided to the stakeholders, and how the system could respond to sensed changes in the emotional state of a patient.

Non-verbal persuasion and communication in an affective agent

by Christopher Edward Peters

Andre, E., Bevacqua, E., Heylen, D., Niewiadomski, R., Poggi, I., Pelachaud, C., Peters, C., and Rehm, M. "Non-verbal persuasion and communication in an affective agent". In P. Petta, C. Pelachaud and R. Cowie (Eds.), Emotion-Oriented Systems: The Humaine Handbook, pp. 585-608, Cognitive Technologies Series, Springer, January 2011 isbn:978-3-642-15183-5
Bibtex avaialble here: http://www.coventry.ac.uk/ec/~cpeters/bibtex/bibtex.html#HandbookPersuasion2011

This chapter deals with the communication of persuasion. Only a small percentage of communication involves words: as... more This chapter deals with the communication of persuasion. Only a small percentage of communication involves words: as the old saying goes, “it’s not what you say, it’s how you say it”. While this likely underestimates the importance of good verbal persuasion techniques, it is accurate in underlining the critical role of non-verbal behaviour during face-to-face communication. In this chapter we restrict the discussion to body language. We also consider embodied virtual agents. As is the case with humans, there are a number of fundamental factors to be considered when constructing persuasive agents. In particular, one who wishes to persuade must appear credible, trustworthy, confident and non-threatening. Knowing how not to behave is also a vital basis for effective persuasion. This includes resolving task constraints or other factors with the social perception considerations. These social virtual agents face many of the same problems as humans have in controlling and expressing themselves in an appropriate manner so as to establish and maintain persuasive interaction. All along the chapter, much of our discussion will handle concepts applicable both to agent and human behaviour.

Expressive copying behavior for socially interacting machines: A perceptual analysis

by Christopher Edward Peters

Castellano, G., Mancini, M., Peters, C., and McOwan, P. W. "Expressive copying behavior for socially interacting machines: A perceptual analysis", IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans, to appear: 2012 doi: 10.1109 / TSMCA.2011.2172415

Successful human interaction commonly involves prototypical exchanges where interactors are engaged, synchronized and... more Successful human interaction commonly involves prototypical exchanges where interactors are engaged, synchronized and harmonious in their behaviors. The copying of aspects of the other’s behavior, at different levels, seems central to establishing and maintaining such empathic connections. Yet many questions remain unanswered, in particular, how it is possible to reflect the same affective content back to the other when the actual motion itself is not exactly the same as theirs. This work presents a perceptual study in which emotional gestures conducted by an actor were mapped onto synthesized versions generated by an embodied virtual agent. Copying is at the expressive level, where qualities such as the fluidity or expansiveness of gestures are considered, rather than exact lowlevel motion matching. Participants were later asked to rate the emotional content of video recordings of both the original and the synthesized gestures. A statistical analysis shows that, in most cases, participants associated the emotional content of the agent’s gestures with that intended to be expressed by the original actor. The results suggest that a combination of the type of movement performed and its quality are important for successfully communicating emotions.

Fundamentals of agent perception and attention modelling

by Christopher Edward Peters

Peters, C., Castellano, G., Rehm, M., Andre, E., Volpe, G., Camurri, A., Raouzaiou, A., Rapantzikos, K., Karpouzis, K., and Vasalou, A. .
In P. Petta, C. Pelachaud and R. Cowie (Eds.), Emotion-Oriented Systems: The Humaine Handbook, pp. 293-319, Cognitive Technologies Series, Springer, January 2011 isbn:978-3-642-15183-5
Bibtex available here:
http://www.coventry.ac.uk/ec/~cpeters/bibtex/bibtex.html#HandbookPerception2011

Perception and attention mechanisms are of great importance for entities situated within complex dynamic environments.... more Perception and attention mechanisms are of great importance for entities situated within complex dynamic environments. With roles extending greatly beyond passive information services about the external environment, such mechanisms actively prioritise, augment and expedite information to ensure that the potentially relevant is made available so appropriate action can take place. Here, we describe the rationale behind endowing artificial entities, or virtual agents, with real-time perception and attention systems. We cover the fundamentals of designing and building such systems. Once equipped, the resulting agents can achieve a more substantial connection with their environment for the purposes of reacting, planning, decision making and, ultimately, behaving.

mFeeling: An affective mobile system that measures people experiences with their local environment

by eiman kanjo

IEEE Pervasive Computing journal , WIP, Imapct Factor 3.08, July 2012

“Affective computing” is how computers recognize and respond to human emotion. While this field remains fairly... more “Affective computing” is how computers recognize and respond to human emotion. While this field remains fairly undeveloped compared to others, sensor technology is improving rapidly in terms of accuracy, usability and cost. It is possible now to connect biological sensors such as GSR (Galvanic Skin Conductors), EEG, temperature and heart rate monitor to computers. Also mobile sensing is increasingly becoming part of everyday life due to the rapid evolution of the mobile phone into a powerful sensing platform.
In this project we will investigate how mobile phones can recognize, interpret and express human emotion and how can we better understand people’s emotional experiences with their surrounding environment or service. Emotions are personal but they can also be shared amongst groups, just as street-passers can feel similar emotions around certain places.

How to Do Multimodal Detection of Affective States?

by Javier Gonzalez Sanchez

Gonzalez-Sanchez, J.;  Christopherson, R.M.;  Chavez-Echeagaray, M.E.;  Gibson, D.C.;  Atkinson, R.;  Burleson, W.; 
Sch. of Comput., Arizona State Univ., Tempe, AZ, USA

This paper appears in: Advanced Learning Technologies (ICALT), 2011 11th IEEE International Conference on

The human-element is crucial for designing and implementing interactive intelligent systems, and therefore on... more The human-element is crucial for designing and implementing interactive intelligent systems, and therefore on instructional design. This tutorial provides a description and hands-on demonstration for detection of affective states and a description of devices, methodologies and tools necessary for automatic detection of affective states. Automatic detection of affective states requires that the computer sense information that is complex and diverse, it can range from brain-waves signals, and biofeedback readings to face-based and gesture emotion recognition to posture and pressure sensing. Obtaining, processing and understanding that information, to create systems that improve learning, requires the use of several sensing devices (and their perceiving algorithms) and the application of software tools.

Looking at the viewer: analysing facial activity to detect personal highlights of multimedia contents

by Jacopo Staiano

The Affective Weight of Lexicon

by Alessandro Valitutti

Developing Affective Lexical Resources

by Alessandro Valitutti

WordNet-Affect: An Affective Extension of WordNet

by Alessandro Valitutti

A computational model of affects

by Mika Turkia

Design of an emotional and social interaction paradigm for the animation of 3D characters: the case of a therapy for brain injured people (the mirror neuron paradigm)

by María Lucila Morales-Rodríguez

We propose an emotional and social interaction paradigm for the behavioural animation of virtual characters. Our... more We propose an emotional and social interaction paradigm for the behavioural animation of virtual characters. Our research focuses on the production of verbal and paraverbal interactions between a human and a virtual character in the context of virtual therapy.
The aim of this project is to assess virtual therapies for the rehabilitation of people with brain injuries caused by cerebral vascular accidents. The therapeutic paradigm is based on the neuroscience concept of ‘‘mirror neurones’’ which emphasise the perceptual side in the process of recovering neural damage. In our paradigm, the patient interacts with a virtual therapist (embodied autonomous agent) that guides and encourages him in
order to fulfil the drills. Furthermore and following the ‘‘mirror neurone paradigm’’, the patient can only see his virtual hands (which are handicapped). The virtual platform provides the patient with a vision of his hands undertaking a movement that he cannot perform.
Doing so, we expect that the perception of this virtual movement may improve the recuperation (mirror neurone paradigm). In this paper, we also introduce the notion of an ‘‘intelligent emotional character’’ to produce a sense of social as well as emotional immersion in order to improve the interaction between the patient and the virtual therapist. We discuss the role of emotional interaction in the context of virtual social interaction and in particular its reflexive dimension.
Finally, we present an architecture for the animation of virtual characters based on a multidisciplinary approach to model the emotional and social interaction.

Towards the Simulation of Social Interactions through Embodied Conversational Agents

by María Lucila Morales-Rodríguez

María Lucila Morales-Rodríguez, Bernard Pavard, Juan J. González B., and José A. Martínez F.
Proceedings of the 3rd International Workshop on Hybrid Artificial Intelligence Systems. Burgos University, Burgos, Spain, September 24-27, 2008.
HYBRID ARTIFICIAL INTELLIGENCE SYSTEMS LNAI 5271 ISSN 0302-9743 pp. 551-557. ISBN 978-3-540-87655-7.

Abstract:
In this paper, we present a pluridisciplinary approach in order to model the behavior of an embodied... more Abstract:
In this paper, we present a pluridisciplinary approach in order to model the behavior of an embodied conversational agent that expresses emotional and social interactions. We present our methodology to reproduce credible social interactions. Particularly, we discuss the role of the context, the culture, and the emotions in the model of the management of speech acts. This model has been implemented in the context of virtual therapy to simulate the interaction between a therapist and a post-CVA patient.

Modèle d'interaction sociale pour des agents conversationnels animés: application à la rééducation de patients cérébro-lésés

by María Lucila Morales-Rodríguez

Thèse de Doctorat, Institut de Recherche en Informatique de Toulouse. Université Paul Sabatier: Toulouse, France. p. 108
2007

Social interaction in virtual universe opens new horizons in the field of information technologies, and particularly,... more Social interaction in virtual universe opens new horizons in the field of information technologies, and particularly, in the use of video games for professional purposes (a.k.a. serious game). Introducing a certain level of realism in the social and emotional interactions of virtual characters makes it possible to consider many professional applications, which could have only been developed until now in the field of human-human interaction.
The main difficulty of this approach is to produce a human interaction dynamic that makes sense for the tasks and actions that must be achieved by the characters. In order to develop this kind of dynamics, we have referred to (i) the approaches of situated cognition (so as to take into account the role of the objects of the environment), and (ii) representational approaches for the production of speech acts.
We focus on the production of verbal, nonverbal and paraverbal interactions between a human and a virtual character in the context of virtual therapy. Initially, we were interested in the description of the dynamics of the visual, gesture and verbal interactions between a patient and a real therapist. This study enabled us to categorize the phases of dialogue and to elaborate a model of the dynamics of the interaction, taking into account the personality of the actors, their intentions, and their verbal and gesture behaviours.
From this model, we have elaborated a generic architecture, which is articulated around a behavioural model and a kinesics model. The behavioural model selects the speech acts and
the attitude of the character (empathic, neutral, etc). The kinesics model selects and integrates the facial, posture and gesture expressions related to the results of the behavioural model.
From a theoretical point of view, we were interested in the concept of presence in order to identify which was the nature of the relevant couplings making it possible to produce a feeling of social presence (i.e. a feeling of living a social interaction not mediated by the elements of the technical device for the human actors which animate their characters).
Finally, the device was used within the framework of a research program on the concept of mirror neurons. The assumption is that the presentation of a virtual image of its handicapped member to the patient would enable him to better recover. For this purpose,
IRMf tests associated with each experiment would allow to confirm or to invalidate the role of these mirror neurons in the process of recovery.

Towards a Gesture Selection Model influenced by Personality and Emotional State for Virtual Characters

by María Lucila Morales-Rodríguez

María Lucila Morales-Rodríguez, Patricia Alanis Maldonado, Claudia Guadalupe Goméz Santillán, Arturo Hernández Ramírez and Rodolfo A. Pazos Rangel.
Special Issue in Advances in Artificial Intelligence and Aplications, Research in Computing Science 51, 2010, pp122-129
Workshop on Hybrid Intelligent System in MICAI 2010, November 8-13, 2010, Pachuca, Hgo. México.

Abstract.
The virtual characters are being used increasingly in user interfaces to improve human-machine... more Abstract.
The virtual characters are being used increasingly in user interfaces to improve human-machine communication. For this reason, it is necessary to improve the interaction of these characters in a similar way like the communication between human beings. This paper presents a general model to characterize and select gestures influenced by personality and emotional state for virtual characters so they can communicate intentions, feelings and ideas.

Emotional Conversational Agents in Clinical Psychology and Psychiatry

by María Lucila Morales-Rodríguez

María Lucila Morales-Rodríguez, Juan Javier González B., Rogelio Florencia Juárez, Hector J. Fraire Huacuja and José A. Martínez Flores
Mexican International Conference on Artificial Intelligence 2010, November 8-13, 2010 Pachuca, Hgo. Mexico.
Advances in Artificial Intelligence, Pages 458-466. Lecture Notes in Computer Science, Volume 6437, 2010

This paper is based on a project at the University of Barcelona to develop the skills to diagnose the Generalized... more This paper is based on a project at the University of Barcelona to develop the skills to diagnose the Generalized Anxiety Disorder (GAD) in students of psychology and psychiatry using a chatbot. The problem we address in this paper is to convert a chatbot in an emotional conversational agent capable of generating a believable and dynamic dialogue in natural language. For it, the dialogues convey traits of personality, emotions and its intensity. We propose to make an AIML language extension for the generation of believable dialogue, this extension will allow to create a more realistic scenario for the student to diagnose the condition simulated by the conversational agent. In order to measure the perception of the emotional state of the ECA expressed by the speech acts a survey was applied.

Lupita: Software Architecture for Virtual Characters on the Internet and Mobile devices

by María Lucila Morales-Rodríguez

José Gildo de Araújo-Júnior, Patricia Alanis-Maldonado, Ma. Lucila Morales-Rodríguez, Claudia Gpe. Goméz-Santillán, Juan Javier González B. and Héctor J. Fraire Huacuja.
Special Issue in Advances in Artificial Intelligence and Aplications, Research in Computing Science 51, 2010, pp165-171
Workshop on Intelligent Learning Environments in MICAI 2010, November 8-13, 2010, Pachuca, Hgo. México.

Abstract.
This paper presents educational software based on a math tutor for elementary school children. This... more Abstract.
This paper presents educational software based on a math tutor for elementary school children. This software teaches the basics operations of mathematics through a virtual tutor endowed with personality. The virtual tutor is a cognitive agent that integrates a behavioral model, verbal and nonverbal expressions. This architecture was based on the concepts of computing Clouding and RIA (Rich Internet Application) for easy access through Internet and mobile devices.

Caracterización y selección de expresiones gestuales influenciadas por la personalidad y el estado emocional para personajes virtuales

by María Lucila Morales-Rodríguez

Patricia Alanis Maldonado, María Lucila Morales-Rodríguez, Memorias del III Encuentro de Investigadores del I.T.C.M.,8-11 Diciembre 2009, Cd. Madero, Tamaulipas.

Los personajes virtuales se están utilizando cada vez más en las interfaces de usuario para mejorar la comunicación... more Los personajes virtuales se están utilizando cada vez más en las interfaces de usuario para mejorar la comunicación hombre-máquina. Por este motivo, surge la necesidad de mejorar las formas de interacción de estos personajes de una manera que se asemeje a las formas de comunicación entre los seres humanos. En este artículo se presenta un modelo para caracterizar y seleccionar las expresiones gestuales asociadas a la personalidad y estado emocional para personajes virtuales que permitan comunicar intenciones, sentimientos e ideas.

Agente Emocional utilizado como un tutor en un juego educativo de matemáticas para niños

by María Lucila Morales-Rodríguez

Rogelio García Rodríguez, Rogelio Florencia Juárez, María Lucila Morales-Rodríguez.
Instituto Tecnológico de Orizaba Noviembre 3-4, 2009, Orizaba, Veracruz. Memorias del Coloquio Nacional de Investigación Multidisciplinaria, pp 311-317

Las últimas evaluaciones de nivel académico realizadas en México a través de la prueba Evaluación Nacional de Logro... more Las últimas evaluaciones de nivel académico realizadas en México a través de la prueba Evaluación Nacional de Logro Académico en Centros Escolares (ENLACE), reflejan que la mayoría de los niños de primaria apenas alcanzan el nivel elemental de conocimiento en la materia de matemáticas.
Interesados en ésta problemática, se desarrolló un software educativo para el reforzamiento del aprendizaje de las operaciones aritméticas básicas. Este software cuenta con un personaje virtual que funge como tutor y que acompaña al niño en el transcurso de todas las actividades del juego. Este personaje incorpora en su comportamiento un modelo emocional haciéndolo más dinámico y realista.
Se realizó un experimento para conocer si en realidad el software desarrollado tiene un impacto positivo en el aprendizaje de las operaciones aritméticas básicas en los niños. Los resultados del experimento, mostraron que los niños que utilizaron el software, obtuvieron mejores calificaciones y tardaron menos tiempo en los exámenes de prueba aplicados que los niños que practicaron las operaciones aritméticas de manera tradicional. Consideramos que el uso de software educativo que incorpore tutores con emociones puede ser una nueva forma de que los niños refuercen los conocimientos y habilidades aprendidas en la escuela.

Virtual Humans and Social Interaction

by María Lucila Morales-Rodríguez

María Lucila Morales-Rodriguez, Bernard Pavard, and Juan J. González B. Proceedings of the 2009 International Conference on Computer Graphics and Virtual Reality. Las Vegas, Nevada,USA, July 13-16, 2009. CSREA Press pp 158-163

Abstract
We thought that, virtual reality cannot merely be reduced to a hardware system, another way to achieve... more Abstract
We thought that, virtual reality cannot merely be reduced to a hardware system, another way to achieve the optimal experience, is to produce a sense of immersion associated to an emotional and social experience inside the virtual environment. We believe that Virtual Characters that express a social and emotional behavior in their interaction could produce a sense of immersion in the user that interact with them. We propose to use this kind of character to develop social and emotional interfaces, capable to produce a believable Social Interaction.