Multisensory Integration

Impairments in Multisensory Processing are Not Universal to the Autism Spectrum: No Evidence for Crossmodal Priming Deficits in Asperger Syndrome

by Nicole David

Co-authored with Till R. Schneider, Kai Vogeley, and Andreas K. Engel

Individuals suffering from autism spectrum disorders (ASD) often show a tendency for detail- or feature-based... more Individuals suffering from autism spectrum disorders (ASD) often show a tendency for detail- or feature-based perception (also referred to as ‘‘local processing bias’’) instead of more holistic stimulus processing typical for unaffected people. This local processing bias has been demonstrated for the visual and auditory domains and there is evidence that multisensory processing may also be affected in ASD. Most multisensory processing paradigms used social-communicative stimuli, such as human speech or faces, probing the processing of simultaneously occuring sensory signals. Multisensory processing, however, is not limited to simultaneous stimulation. In this study, we investigated whether multisensory processing deficits in ASD persist when semantically complex but nonsocial stimuli are presented in succession. Fifteen adult individuals with Asperger syndrome and 15 control persons participated in a visual-audio priming task, which required the classification of sounds that were either primed by semantically congruent or incongruent preceding pictures of objects. As expected, performance on congruent trials was faster and more accurate compared with incongruent trials (crossmodal priming effect). The Asperger group, however, did not differ significantly from the control group. Our results do not support a general multisensory processing deficit, which is universal to the entire autism spectrum.

Bayesian models for multimodal perception of 3D structure and motion

by João Filipe Ferreira

Serino A, Canzoneri E, Avenanti A (2011). Fronto-parietal areas necessary for a multisensory representation of peripersonal space in humans: an rTMS study. Journal of Cognitive Neuroscience 23, 2956-2967.

by Alessio Avenanti

A network of brain regions including the ventral premotor cortex (vPMc) and the posterior parietal cortex (PPc) is... more A network of brain regions including the ventral premotor cortex (vPMc) and the posterior parietal cortex (PPc) is consistently recruited during processing of multisensory stimuli within peripersonal space (PPS). However, to date, information on the causal role of these fronto-parietal areas in multisensory PPS representation is lacking. Using low-frequency repetitive TMS (rTMS; 1 Hz), we induced transient virtual lesions to the left vPMc, PPc, and visual cortex (V1, control site) and tested whether rTMS affected audio-tactile interaction in the PPS around the hand. Subjects performed a timed response task to a tactile stimulus on their right (contralateral to rTMS) hand while concurrent task-irrelevant sounds were presented either close to the hand or 1 m far from the hand. When no rTMS was delivered, a sound close to the hand reduced RT-to-tactile targets as compared with when a far sound was presented. This space-dependent, auditory modulation of tactile perception was specific to a hand-centered reference frame. Such a specific form of multisensory interaction near the hand can be taken as a behavioral hallmark of PPS representation. Crucially, virtual lesions to vPMc and PPc, but not to V1, eliminated the speeding effect due to near sounds, showing a disruption of audio-tactile interactions around the hand. These findings indicate that multisensory interaction around the hand depends on the functions of vPMc and PPc, thus pointing to the necessity of this human fronto-parietal network in multisensory representation of PPS.

Multisensory enhancement of attentional capture in visual search

by Pawel J. Matusz

Multisensory Information Facilitates Reaction Speed by Enlarging Activity Difference between Superior Colliculus Hemispheres in Rats

by Miquel Bosch

Animals can make faster behavioral responses to multisensory stimuli than to unisensory stimuli. The superior... more Animals can make faster behavioral responses to multisensory stimuli than to unisensory stimuli. The superior colliculus (SC), which receives multiple inputs from different sensory modalities, is considered to be involved in the initiation of motor responses. However, the mechanism by which multisensory information facilitates motor responses is not yet understood. Here, we demonstrate that multisensory information modulates competition among SC neurons to elicit faster responses. We conducted multiunit recordings from the SC of rats performing a two-alternative spatial discrimination task using auditory and/or visual stimuli. We found that a large population of SC neurons showed direction-selective activity before the onset of movement in response to the stimuli irrespective of stimulation modality. Trial-by-trial correlation analysis showed that the premovement activity of many SC neurons increased with faster reaction speed for the contraversive movement, whereas the premovement activity of another population of neurons decreased with faster reaction speed for the ipsiversive movement. When visual and auditory stimuli were presented simultaneously, the premovement activity of a population of neurons for the contraversive movement was enhanced, whereas the premovement activity of another population of neurons for the ipsiversive movement was depressed. Unilateral inactivation of SC using muscimol prolonged reaction times of contraversive movements, but it shortened those of ipsiversive movements. These findings suggest that the difference in activity between the SC hemispheres regulates the reaction speed of motor responses, and multisensory information enlarges the activity difference resulting in faster responses.

Viewing Geometry Determines How Vision and Haptics Combine in Size Perception

by Sergei Gepshtein

Co-authored with M. S. Banks, published in Current Biology, 2003

Vision and haptics have different limitations and advantages because they obtain information by different methods. If... more Vision and haptics have different limitations and advantages because they obtain information by different methods. If the brain combined information from the two senses optimally, it would rely more on the one providing more precise information for the current task. In this study, human observers judged the distance between two parallel surfaces in two within-modality experiments (vision-alone and haptics-alone) and in an intermodality experiment (vision and haptics together). In the within-modality experiments, the precision of visual estimates varied with surface orientation, as expected from geometric considerations; the precision of haptic estimates did not. An ideal observer that combines visual and haptic information weights them differently as a function of orientation. In the intermodality experiment, humans adjusted visual and haptic weights in a fashion quite similar to that of the ideal observer. As a result, combined size estimates are finer than is possible with either vision or haptics alone; indeed, they approach statistical optimality.

Smoking experience modulates the cortical integration of vision and haptics

by Yavor Yalachkov

published in Neuroimage

Perceived Touch Location is Coded using a Gaze Signal

by Lisa Pritchett

My first publication with my supervisor Laurence Harris. It will appear in a special issue of Experimental Brain Research on multisensory processing.

The location of a touch to the skin, first coded in body coordinates, may be transformed into retinotopic coordinates... more The location of a touch to the skin, first coded in body coordinates, may be transformed into retinotopic coordinates to facilitate visual-tactile integration. In order for the touch location to be transformed into a retinotopic reference frame, the location of the eyes and head must be taken into account. Previous studies have found eye position-related errors (Harrar and Harris in Exp Brain Res 203:615-620, 2009) and head position-related errors (Ho and Spence Brain Res 1144:136-141, 2007) in tactile localization, indicating that imperfect versions of eye and head signals may be used in the body-to-visual coordinate transformation. Here, we investigated the combined effects of head and eye position on the perceived location of a mechanical touch to the arm. Subjects reported the perceived position of a touch that was presented while their head was positioned to the left, right, or center of the body and their eyes were positioned to the left, right, or center in their orbits. The perceived location of a touch shifted in the direction of both head and the eyes by approximately the same amount. We interpret these shifts as being consistent with touch location being coded in a visual reference frame with a gaze signal used to compute the transformation.

Visuohaptic convergence in a corticocerebellar network

by Yavor Yalachkov

published in European Journal of Neuroscience

Investigating human audio-visual object perception with a combination of hypothesis-generating and hypothesis-testing fMRI analysis tools

by Jasper van den Bosch

Touching Sounds: Thalamocortical Plasticity and the Neural Basis of Multisensory Integration

by Jasper van den Bosch