Download (.pdf) Multimodal analyses identify linked functional and white matter abnormalities within the working memory network in schizophrenia
Sugranyes G, Kyriakopoulos M, Dima D, O'Muircheartaigh J, Corrigall R, Pendelbury G, Hayes D, Calhoun VD, Frangou S
Schizophr
BACKGROUND: Dysconnectivity between brain regions is thought to underlie the cognitive abnormalities that characterise... more
BACKGROUND: Dysconnectivity between brain regions is thought to underlie the cognitive abnormalities that characterise schizophrenia (SZ). Consistent with this notion functional magnetic resonance imaging (fMRI) and diffusion tensor imaging (DTI) studies in SZ have reliably provided evidence of abnormalities in functional integration and in white matter connectivity. Yet little is known about how alterations at the functional level related to abnormalities in anatomical connectivity.
METHODS: We obtained fMRI data during the 2-back working memory task from 25 patients with SZ and 19 healthy controls matched for age, sex and IQ. DTI data were also acquired in the same session. In addition to conventional unimodal analyses we extracted "features" [contrast maps for fMRI and fractional anisotropy (FA) for DTI] that were subjected to joint independent component analysis (JICA) in order to examine interactions between fMRI and DTI data sources.
RESULTS: Conventional unimodal analyses revealed both functional and structural deficits in patients with SZ. The JICA identified regions of joint, multimodal brain sources that differed in patients and controls. The fMRI source implicated regions within the anterior cingulate and ventrolateral prefrontal cortex and in the cuneus where patients showed relative hypoactivation and within the frontopolar cortex where patients showed relative hyperactivation. The DTI source localised reduced FA in patients in the splenium and posterior cingulum.
CONCLUSIONS: This study promotes our understanding of structure-function relationships in SZ by characterising linked functional and white matter changes that contribute to working memory dysfunction in this disorder.
Following in Rabelais' Footsteps: Immersive History and the 3D Virtual Buildings Project
by John Bonnett
Journal of the Association for History and Computing, vol. 6, no. 2, September 2003
Article was subsequently published at journal's request in:
History and Computing. 13(2): 107-150. 2001 (Published 2004).
In Europe and North America, scholars are beginning to shed the assumptions of print culture. They are also only... more In Europe and North America, scholars are beginning to shed the assumptions of print culture. They are also only beginning to perceive the possibilities of a computerized, electro-magnetic culture. In this first decade of the twenty-first century, historians face a challenge: how, and to what extent, are we to appropriate the 3D-immersive environment as an instrument of representation and interaction? In many ways, the situation is analogous to that faced by scholars at the close of the Roman Empire. Virtual Reality (VR) - the generation of artificial, three-dimensional, immersive environments by the computer - is the codex of our time. As a medium of communication, it presents new methods for representation, narration and instruction. In this study I suggest how using the 3D Virtual Buildings Project as a framework for discussing what is currently possible and as a prototype for discussing what will soon be possible.
From Embodied Consciousness to Socially Embedded Metaphors and Multimodal Cognitive Semiotics: Gesture, Speech and Artwork of Christian Street …
by Vito Evola
Presented at the 5° Convegno Nazionale dell’Associazione Italiana di Scienze Cognitive (AISC): “Mente e Cervello” - Torino, 30 Nov. - 2 Dec., 2008; Talk: “Verbum Incarnatum: Dalla coscienza embodied alle metafore socialmente embedded e la semiotica cognitiva multimodale. Discorsi, gestualità e disegni di predicatori di strada cristiani e di un satanista. (tr. Verbum Incarnatum: From embodied consciousness to socially embedded metaphors and multimodal cognitive semiotics. Speech, gesture and artwork of Christian street preachers and a Satanist)”
See also "Multimodal Semiotics of Spiritual Experiences: Representing Beliefs, Metaphors, and Actions", in... more See also "Multimodal Semiotics of Spiritual Experiences: Representing Beliefs, Metaphors, and Actions", in Fey Parrill, Vera Tobin, and Mark Turner (eds.) (2010). Form, Meaning, and Body. Stanford: CSLI. pp. 41-60.
Semiótica Multimodal de las Experiencias Espirituales: La Representación de Creencias, Metáforas y Acciones
by Vito Evola
Evola, Vito (2009). “Semiótica Multimodal de las Experiencias Espirituales: La Representación de Creencias, Metáforas y Acciones”. In Carmen M. Bretones et al. (eds). Applied Linguistics Now: Understanding Language and Mind / La Lingüística Aplicada Hoy: Comprendiendo el Lenguaje y la Mente. Almería: Universidad de Almería. pp. 1571-1586
Los textos islámicos enseñan que cualquier manera de hablar de Dios no es sino una metáfora, es decir, una manera de... more
Los textos islámicos enseñan que cualquier manera de hablar de Dios no es sino una metáfora, es decir, una manera de hablar acerca de Dios de tal modo que el ser humano pueda comprender fácilmente (cf. Bausani, 1980: 16-17). En este sentido, se trata de metáforas conceptuales tal como son explicadas por la Lingüística Cognitiva: cuando el Qur´an, o la Biblia, u otros textos sagrados hablan de la Divinidad en términos humanos, usan metáforas, y se recurre esencialmente a la metáfora conceptual de alto nivel ABSTRACTO ES CONCRETO.
¿Cómo se relacionan los creyentes y practicantes de hoy, y no solamente los teólogos, con ciertas metáforas transmitidas por su fe?, y ¿qué pueden decirnos dichas metáforas sobre los conceptos que los individuos tienen sobre sí mismos? Por otra parte, ¿de qué manera mantiene el creyente una representación viable de sí mismo y del mundo que le rodea a pesar de los aspectos aparentemente contradictorios de sus representaciones?
En este artículo intento mostrar que la Lingüística Cognitiva puede ofrecer una nueva perspectiva en el tratamiento de estas cuestiones, y que las metáforas que se refieren a los conceptos “más significativos”, como el concepto de persona y el de lo transcendente, están profunda y firmemente enraizadas en nuestro sistema conceptual individual, poniendo en evidencia asociaciones metafóricas y metonímicas que a menudo no son del todo evidentes.
Can we explain cross-modal representation with neural algorithms alone? – a comment on Paillard.
by Mark Elliott
Elliott, M. A. (1999). Can we explain cross-modal representation with neural algorithms alone? – a comment on Paillard. In G. Aschersleben, T. Bachmann & J. Müsseler (Eds.). Cognitive Contributions to the Perception of Spatial and Temporal Events. Amsterdam Elsevier, (pp.117-119).
Paillard’s chapter “Motor determinants of a unified perceived world” highlights a set of intriguing possibilities:... more
Paillard’s chapter “Motor determinants of a unified perceived world” highlights a set of intriguing possibilities: firstly there are constraints on the quality of visual (and one might also assume auditory) perception that are imposed by somatosensory and positive/negative taxic experiences in the environment. In addition, and by extension, Paillard cites research by Collins and Bach-y-Rita, (1973), which indicates that, for at least one level of description, stimulus representations may be ‘qualitatively identical’ irrespective to the sensory modality through which the stimulus was processed. Thus, and as Paillard describes, the sensation derived from ‘feeling’ the representation of a visual image is able to guide definition of the figural properties of the image, the level of ‘representational commonality’ being the representation of the spatial configuration of the image. Recent research in both psychophysics and electrophysiology has revealed a number of examples in which behaviour is guided by (i.e. Spence & Driver, 1996), and brain activity has been argued to correlate with, the formation of a multimodal representation (although accounts are not in agreement as to the cortical origin of this activity: see Giard & Peronnet, 1998; Hackley & Valle-Incián, 1998; Schröger & Widmann, 1998). However the question remains as to the best level of description for this representation. Phillips and Singer (1997) consider this point in some depth: the formation of representations which may be described in terms of a common cortical algorithm (for example the commonality of certain neuronal firing frequencies during auditory or visual stimulus coding), are not sufficiently described by appeal to that same neuronal algorithm. Thus, even if we assume that a given representational state (for example, that which results from slowly increasing the temperature of a thermoconductive pad attached to the skin) correlates with some definable property of neural activity (such as the corresponding increase in 40-Hz activity across the somatosensory cortex [Chen & Herrmann, in progress, Herrmann, personal communication]), the statistical relationship between the 40-Hz activity and the associated representation of ‘heat’ is unlikely to completely describe the experience of ‘hotness’, i.e. the ‘qualia’ associated both with the representation and the 40-Hz activity in the brain.
In addition, the idea that multi-modal representations may include common property states (that contribute to the specification of the quale) also bears relation to the ‘context’ of a given representation (i.e. in what circumstances does a given representation become manifest and for what reasons do we have representations with specific or shared properties in the first place). The question of context is most certainly not trivial and is as broad as it is complex. For example, one could ask an ethologist about the origin of associated representations such as ‘stripes’, the colours ‘yellow’ and ‘black’ with the behaviour schema ‘don’t eat this one it’s poisonous’ (i.e. amongst some birds), and the relationship of such representations with the (desired) existential context ‘I’m poisonous therefore not edible’ (i.e. amongst some bees). It is unlikely that either the common association or common context shared by these representations can be explained simply in terms of one or a number of neural algorithms, or for that matter, exclusively in terms of the representations themselves. The ethologist might argue that these associations, the consequent behaviour, behavioural context and the selection of the gene that produces this colouration have resulted from innumerable interactions of predator and prey during the process of evolution. In addition, and given that the collection of representations (above) instantiated within the bird brain are, to some extent, responsible for the peaceful co-existence of the birds and the bees, the ‘fitness’ potential of black and yellow stripes has been exploited to full potential by certain species of ‘non-poisonous’ flying insects. On this basis, one is forced to ask whether the environment itself (i.e. the amount of ‘bogus’ black and yellow striped insects) is subject to alteration by virtue of some statistical structure that is best understood at the level of the representation and of the reciprocity of this relationship. Clearly, the cortical algorithm alone is not sufficient to describe a pattern of commonality (or identity) between the qualitative content of two or more representations, or to sufficiently explain the ‘contexts’ of that commonality. Thus, a coherent description, with explanatory power, is neither the exclusive province of a single level of analysis and almost certainly not the ultimate result of reducing the overall level of analysis to that of systems neuroscience.
However, for current purposes, it is of principal importance that a level of representational commonality exists, which might be adequately described in terms of the statistical structure adopted by patterns of activity in the nervous system. The importance of this possibility relates to the possibility that a psychophysical description of the common property states of a ‘representation’ might also be obtained in terms of statistical structures that emerge from careful and complementary experimentation involving both psychophysical and electrophysiological methods. Penultimately, and as Phillips and Singer (1997) argue (and I agree) both the neurophysical and psychophysical structures evolved directly from, and as natural adaptations to, the statistical structure of the environment and thus we have a third dimension for consideration, namely the information properties of stimuli in the environment. By Paillard’s account extraction of these statistical properties may be conceived of in terms of a common cortical algorithm that generates a representation, independent of the sensory system from which it was derived and which maintains a high level of informational content relatively independently of the modality by which it was encoded. Thus, and on the basis of this notion of representational commonality, Paillard’s account introduces the basis for a prospective methodology, which would certainly permit a direct comparison (or bridging law) to be established between complementary levels of description, the results of which could only advance our current thinking about what a ‘representation’ may or may not be.
References:
Collins, C. C., & Bach-y-Rita, P. (1973). Transmission of pictorial information through the skin. Advances in Biological and Medical Physics, 14, 285-315.
Giard, M. H., & Peronnet, F. (1998). Auditory-visual integration during multimodal object recognition in humans: A behavioral and electrophysiological study. Journal of Cognitive Neuroscience (in press).
Hackley, S. A., & Valle-Incián, F. (1998). Automatic alerting does not speed late motoric processes in a reaction time task. Nature, 391, 786-788.
Phillips, W. A., & Singer, W. (1997). In search of a common foundation for cortical computation. Behavioral and Brain Sciences, 20, 657-722.
Schröger, E., & Widmann, A. (1998). Speeded responses to audio-visual signal changes result from bimodal integration. Psychophysiology, 35, 755-759.
Spence, C., & Driver, J. (1996). Audiovisual links in endogenous covert spatial attention. Journal of Experimental Psychology: Human Perception & Performance, 22, 1005-1030.
On the Variability of the McGurk Effect: Audiovisual Integration Depends on Prestimulus Brain States
by Julian Keil
The McGurk effect demonstrates the influence of visual cues on auditory perception. Mismatching information from both... more The McGurk effect demonstrates the influence of visual cues on auditory perception. Mismatching information from both sensory modalities can fuse to a novel percept that matches neither the auditory nor the visual stimulus. This illusion is reported in 60–80% of trials. We were interested in the impact of ongoing brain oscillations—indexed by fluctuating local excitability and interareal synchronization—on upcoming perception of identical stimuli. The perception of the McGurk effect is preceded by high beta activity in parietal, frontal, and temporal areas. Beta activity is pronounced in the left superior temporal gyrus (lSTG), which is considered as a site of multimodal integration. This area is functionally (de)coupled to distributed frontal and temporal regions in illusion trials. The disposition to fuse multisensory information is enhanced as the lSTG is more strongly coupled to frontoparietal regions. Illusory perception is accompanied by a decrease in poststimulus theta-band activity in the cuneus, precuneus, and left superior frontal gyrus. Event-related activity in the left middle temporal gyrus is pronounced during illusory perception. Thus, the McGurk effect depends on fluctuating brain states suggesting that functional connectedness of left STS at a prestimulus stage is crucial for an audiovisual percept.