Intelligent Systems

Tell me more? The effects of mental model soundness on personalizing an intelligent agent.

by Todd Kulesza

In the proceedings of CHI 2012. Honorable mention for Best Paper award.

What does a user need to know to productively work with an intelligent agent? Intelligent agents and recommender... more What does a user need to know to productively work with an intelligent agent? Intelligent agents and recommender systems are gaining widespread use, potentially creating a need for end users to understand how these systems operate in order to fix their agent's personalized behavior. This paper explores the effects of mental model soundness on such personalization by providing structural knowledge of a music recommender system in an empirical study. Our findings show that participants were able to quickly build sound mental models of the recommender system's reasoning, and that participants who most improved their mental models during the study were significantly more likely to make the recommender operate to their satisfaction. These results suggest that by helping end users understand a system's reasoning, intelligent agents may elicit more and better feedback, thus more closely aligning their output with each user's intentions.

Why-Oriented End-User Debugging of Naive Bayes Text Classification

by Todd Kulesza

Published in ACM Transactions on Interactive Intelligent Systems, Vol. 1, No. 1, October 2011.

Machine learning techniques are increasingly used in intelligent assistants, that is, software targeted at and... more Machine learning techniques are increasingly used in intelligent assistants, that is, software targeted at and continuously adapting to assist end users with email, shopping, and other tasks. Examples include desktop SPAM filters, recommender systems, and handwriting recognition. Fixing such intelligent assistants when they learn incorrect behavior, however, has received only limited attention. To directly support end-user “debugging” of assistant behaviors learned via statistical machine learning, we present a Why-oriented approach which allows users to ask questions about how the assistant made its predictions, provides answers to these “why” questions, and allows users to interactively change these answers to debug the assistant’s current and future predictions. To understand the strengths and weaknesses of this approach, we then conducted an exploratory study to investigate barriers that participants could encounter when debugging an intelligent assistant using our approach, and the information those participants requested to overcome these barriers. To help ensure the inclusiveness of our approach, we also explored how gender differences played a role in understanding barriers and information needs. We then used these results to consider opportunities for Why-oriented approaches to address user barriers and information needs.

Interactive environment by narrative playmates toys

by eiman kanjo

ACM, USA, Journal Paper Volume: 23 Issue: 2 Published: Aug. 2002

Eiman Kanjo, Peter Astheimer

Narration is an important part of play. Toys inspire children to imagine stories. Adding the power of the computer to... more Narration is an important part of play. Toys inspire children to imagine stories. Adding the power of the computer to toys environment can make playing time more cognitive as well as more entertaining. This paper describes our work in developing physical human-computer interface, which merges ordinary children's playsets with computer that can see via a Webcam in order to enhance children's playing technique. The proposed interface with the tracking technology and narrative control can turn a normal child's playset into a fantasyland where children can sense of their world in ways that were not feasible up to date

Vision-Based Innovative Network Games

by eiman kanjo

The Pervasive Fridge

by José Rouillard

Exploring Research Data Interactively. Theme One : A Program of Inquiry

by Jon Awbrey

Awbrey, J.L., and Awbrey, S.M. (August 1990), “Exploring Research Data Interactively. Theme One : A Program of Inquiry”, Proceedings of the Sixth Annual Conference on Applications of Artificial Intelligence and CD-ROM in Education and Training, Society for Applied Learning Technology, Washington, DC, pp. 9–15.

If computer programs were smarter, they would, like people, recognize sequences of events, form models of their... more If computer programs were smarter, they would, like people, recognize sequences of events, form models of their environment, and formulate rules based on experience.  This paper describes the development of a program designed to address the difficult computational problems involved in integrating the inductive and deductive reasoning necessary to perform such tasks.  “Theme One” is a prototype program composed of “Index”, a learning algorithm for sequential data, and “Study”, an algorithm for building logical models.  The project goal is an interactive research tool that assists students and investigators in the exploration of qualitative data using artificial intelligence.

An Architecture for Inquiry : Building Computer Platforms for Discovery

by Jon Awbrey

Awbrey, S.M., and Awbrey, J.L. (May 1991), “An Architecture for Inquiry : Building Computer Platforms for Discovery”, Proceedings of the Eighth International Conference on Technology and Education, Toronto, Canada, pp. 874–875.

More and more we hear the complaint that the gap between research and instruction is widening and a vital sense of... more More and more we hear the complaint that the gap between research and instruction is widening and a vital sense of motivation is falling between the cracks. It is our vision that intelligent computing systems will become a partner in the reintegration of discovery and learning within the inquiry process. We will address certain issues that must be faced if computer media are to have the characteristics necessary to support this integration. The development of the computer to date has required a careful attention to the syntax and semantics of the rather limited symbol systems we have induced them to use. A capacity for communicating in multiple modalities with non-uniform communities of symbol users — for sharing in the discovery of a pluralistic universe — will demand a quantum leap in our understanding of the pragmatic dimensions of symbol use. In the future the capacity for inquiry must permeate the living architecture of the computer system. A computer program that begins to embody these ideas will be discussed.

Reinforcement learning in multiresolution object recognition (2004)

by Taufiq Widjanarko

Proceedings of The 2004 IEEE International Joint Conference on Neural Networks (IJCNN 2004), Vol.2, pp. 1085-1090, 25-29 July 2004

In this work, we propose an adaptive automatic target recognition (ATR) technique that exploits reinforcement learning... more In this work, we propose an adaptive automatic target recognition (ATR) technique that exploits reinforcement learning (RL) for multiresolution object recognition. The RL structure is the implementation of neuro-dynamic programming (NDP) for the critic and action networks. The critic network calculates the cost to-go function J* based on a simplistic ATR plant that involves multiresolution images as the input state variable. The calculation of this function, J* includes the role of the reinforcement signal in the critic network. Output of this critic stage is fed back to update the weights of both action and critic etworks respectively. Our simulation results suggest that RL ay be successfully integrated into an adaptive multiresolution ATR ramework.

An Approach for Developing Intelligent Systems for Sentiment Analysis over Social Networks

by Andrea Bellucci

Enhancing Sound Design with Conceptual Blending of Sound Descriptors

by Prof Eduardo R. Miranda

Integrating perceptual signal features within a multi-facetted conceptual model for automatic image retrieval

by Mohammed Belkhatir

Combining visual semantics and texture characterizations for precision-oriented automatic image retrieval

by Mohammed Belkhatir

Towards a High-Level Audio Framework for Video Retrieval Combining Conceptual Descriptions and Fully-Automated Processes

by Mohammed Belkhatir

The growing need for 'intelligent' video retrieval systems leads to
new architectures combining multiple... more The growing need for 'intelligent' video retrieval systems leads to
new architectures combining multiple characterizations of the video content that rely on highly expressive frameworks while providing fully-automated indexing and retrieval processes. As a matter of fact, addressing the problem of combining modalities within expressive frameworks for video indexing and retrieval is of huge importance and the only solution for achieving significant retrieval performance.
This paper presents a multi-facetted conceptual framework integrating multiple characterizations of the audio content for automatic video retrieval. It relies on an expressive representation formalism handling high-level audio descriptions of a video document and a full-text query framework in an attempt to operate video indexing and retrieval on audio features beyond state-of-the-art architectures operating on low-level features and keyword-annotation frameworks. Experiments on the multimedia topic search task of the TRECVID 2004 evaluation campaign validate our proposal.

An Evolutionary Approach to Skillful Motion Control of Industrial Robots (PhD thesis)

by Thrishantha Nanayakkara

This thesis addresses issues related to evolution of skillful motions of industrial robots. It is motivated by the... more This thesis addresses issues related to evolution of skillful motions of industrial robots. It is motivated by the increasing need to have more intelligent and autonomous robots to improve the flexibility and robustness of factory automation systems in uncertain work environments. The thesis focuses on two problems: 1) evolutionary optimization in changing fitness landscapes and environments, and 2) synthesis of skillful behavior in robots, based on natural phenomena in motor skill acquisition together with the implications of human morphology to skillful manipulation. A considerable weight is given to learning from nature and its robust dynamics to evolve smart species like humans and human-like manipulation skills.

The first chapter gives an introduction of the background, motivation and the overview of the thesis. The second chapter presents a novel evolutionary algorithm to optimize both in static as well as dynamic multi-modal fitness landscapes. The characteristic features of how the nature sustains a robust evolutionary process in changing environmental conditions are discussed and modeled in a simple way. It is shown through mathematical and empirical evidences that the new evolutionary algorithm performs better than conventional methods.

Redundant manipulators are best suited for human-like manipulation tasks due to their morphological similarity with the human arm. Yet, identification of dynamics and model-based control of redundant manipulators is a significant problem due to the volume of calculations and the resulting processing burden. In the third chapter, this thesis presents a very effective alternative, based on Runge--Kutta--Gill neural networks (RKGNNs). The generalizability of the adopted design principles makes it very effective in real world applications. Furthermore, the mechanism found in the human motor system to learn complex skills by adaptive combination of motor primitives is treated as a basis to develop a novel model based control method for multi-link manipulators in the fourth chapter.

Based on the view that intelligence is an effect of some underlying causes and conditions in the biology of the brain, the thesis hypothesizes that the level of motor skills of a human is as strong as the comprehensiveness of the internal evaluation functions pertaining to state dependant actions. In the fifth chapter, this thesis emphasizes the importance of constructing a comprehensive set of multi-objective evaluation functions to evolve skillful behaviors of a robot. The idea is demonstrated through an application of a mobile robot navigating in an uncertain environment.

An important part of intelligence related to human arm motions is that humans make full use of the redundancy of the arm while performing skillful tasks. In the sixth chapter, this thesis presents a novel configuration controller optimized using an evolutionary approach for redundancy resolution of manipulators. A {\em gradually growing} multi-objective fuzzy set based objective function is designed so that it prioritizes objectives and starts with high priority objectives and goes on adding auxiliary objectives over the generations of the evolutionary process. This kind of a gradually growing multi-objective problem (MOP) is supposed to be very similar to the natural process of evolving complex systems starting from simple structures. In the seventh chapter, the importance of this approach to extend the dexterity and reliability of teleoperated manipulators is experimentally demonstrated.

Finally a discussion on the advantages, limitations and drawbacks of the proposed methods is given in the eighth chapter. The experiments to demonstrate the validity of the proposed methods were performed on an industrial 7-DOF manipulator called PA-10, manufactured by the Mitsubishi Heavy Industries Ltd.

Thrishantha Nanayakkara, Keigo Watanabe, Kazuo Kiguchi and Kiyotaka Izumi, "Fuzzy Self-Adaptive RBF Neural Network Based Control of a Seven-Link Industrial Robot Manipulator," in Journal of Advanced Robotics, vol. 15, no. 1, pp. 17-43, 2001.

by Thrishantha Nanayakkara

This paper proposes a method for the identification of dynamics and control of a multi-link industrial robot... more This paper proposes a method for the identification of dynamics and control of a multi-link industrial robot manipulator using Runge-Kutta-Gill neural networks (RKGNNs). RKGNNs are used to identify an ordinary differential equation of the dynamics of the robot manipulator. A structured function neural network (NN) with sub-networks to represent the components of the dynamics is used in the RKGNNs. The sub-networks consist of shape adaptive radial basis function (RBF) NNs. An evolutionary algorithm is used to optimize the shape parameters and the weights of the RBFNNs. Due to the fact that the RKGNNs can accurately grasp the changing rates of the states, this method can effectively be used for long-term prediction of the states of the robot manipulator dynamics. Unlike in conventional methods, the proposed method can even be used without input torque information because a torque network is part of the functional network. This method can be proposed as an effective option for the dynamics identification of manipulators with high degrees-offreedom, as opposed to the derivation of dynamic equations and making additional hardware changes as in the case of statistical parameter identification such as linear least-squares method. Experiments were carried out using a seven-link industrial manipulator. The manipulator was controlled for a given trajectory, using adaptive fuzzy selection of nonlinear dynamic models identified previously. Promising experimental results are obtained to prove the ability of the proposed method in capturing nonlinear dynamics of a multi-link manipulator in an effective manner.

Intelligent Informatics Platform for Nano-Agriculture

by Nirav Ajmeri

The application of nanotechnology in the agricultural sector is likely to facilitate and frame the next stage of... more The application of nanotechnology in the agricultural sector is likely to facilitate and frame the next stage of development of genetically modified crops, precision farming techniques (remote and local sensing), remediation (water treatment plants, pesticide removal from ground water), nano-sensors, nano-agricultural chemicals and most importantly designing smart delivery systems for nutrients and pesticides[1]. Although most of these applications are still in their infancy, they have a great potential to revolutionalize the entire agricultural value chain [2]. The wide spectrum of applications has resulted into emergence of multiple stakeholders such as nano-agriculture researchers, practitioners (agriculturists/ farmers), manufacturers and regulatory bodies. They would be seeking and using knowledge in this nascent area from different perspectives such as research and technology, consumer safety, environmental impact and ethical, legal and social implications. No informatics platform exists to cater to the knowledge needs of various stakeholders in this field. To address this gap, we have developed an intelligent Nano-Agriculture Informatics System (NAIS), wherein these stakeholders can carry out multiple activities of their interest. NAIS incorporates a collaborative and semantically guided process to facilitate knowledge-based activities.

Intelligent Control Systems with an Introduction to System of Systems Engineering by Thrishantha Nanayakkara, Ferat Sahin, and Mo Jamshidi (Hardcover - 15 Oct 2009), publisher: CRC Press, Taylor and Fracis group

by Thrishantha Nanayakkara

Co authored with Mo Jamshidi (former advisor to NASA) and Ferat Sahin

This book integrates the fundamentals of computational intelligence and systems control in a framework applicable to... more This book integrates the fundamentals of computational intelligence and systems control in a framework applicable to both simple dynamic systems and large-scale system of systems (SoS). For decades, NASA has used SoS methods, and major manufacturers—including Boeing, Lockheed-Martin, Northrop-Grumman, Raytheon, BAE Systems—now make large-scale systems integration and SoS a key part of their business strategies, dedicating entire business units to this remarkably efficient approach. This book is an aid to train engineers to integrate traditional systems control theory with soft computing techniques and emerging SoS technology.

Improvement User Interface in Virtual Learning Environment

by Siamak Barzegar

Fixing the Program My Computer Learned: Barriers for End Users, Challenges for the Machine

by Todd Kulesza

Published in the proceedings of IUI 2009.

The results of a machine learning from user behavior can be thought of as a program, and like all programs, it may... more The results of a machine learning from user behavior can be thought of as a program, and like all programs, it may need to be debugged. Providing ways for the user to debug it matters, because without the ability to fix errors users may find that the learned program's errors are too damaging for them to be able to trust such programs. We present a new approach to enable end users to debug a learned program. We then use an early prototype of our new approach to conduct a formative study to determine where and when debugging issues arise, both in general and also separately for males and females. The results suggest opportunities to make machine-learned programs more effective tools.

Toward End-User Debugging of Machine-Learned Classifiers

by Todd Kulesza

From the VL/HCC 2010 Graduate Consortium.

Many machine-learning algorithms learn rules of behavior from individual end users, such as task- oriented desktop... more Many machine-learning algorithms learn rules of behavior from individual end users, such as task- oriented desktop organizers and handwriting recognizers. These rules form a generated “program” tailored specifically to the behaviors of that end user, telling the computer what to do when future inputs arrive. Researchers, however, have only recently begun to explore how an end user can debug these programs when they make mistakes. We present our progress toward enabling end users to test and debug learned programs so that everyone can benefit from intelligent programs adapted to their specific tasks and situations.