BackgroundAttention deficit hyperactivity disorder is clinically diagnosed; however, quantitative analysis to statistically analyze the symptom severity of children with ADHD via the measurement of head movement is still in progress. Studies focusing on the cues that may influence the attention of children with ADHD in classroom settings, where children spend a considerable amount of time, are relatively scarce. Virtual reality allows real-life simulation of classroom environments and thus provides an opportunity to test a range of theories in a naturalistic and (...) controlled manner. The objective of this study was to investigate the correlation between participants’ head movements and their reports of inattention and hyperactivity, and to investigate how their head movements are affected by different social cues of different sensory modalities.MethodsThirty-seven children and adolescents with and without ADHD were recruited for this study. All participants were assessed for diagnoses, clinical symptoms, and self-reported symptoms. A virtual reality-continuous performance test was conducted under four conditions: control, no-cue, visual cue, and visual/audio cue. A quantitativecomparison of the participants’ head movements was conducted in three dimensions using a head-mounted display in a VR classroom environment. Task-irrelevant head movements were analyzed separately, considering the dimension of movement needed to perform the VR-CPT.ResultsThe magnitude of head movement, especially task-irrelevant head movement, significantly correlated with the current standard of clinical assessment in the ADHD group. Regarding the four conditions, head movement showed changes according to the complexity of social cues in both the ADHD and healthy control groups.ConclusionChildren and adolescents with ADHD showed decreasing task-irrelevant movements in the presence of social stimuli toward the intended orientation. As a proof-of-concept study, this study preliminarily identifies the potential of VR as a tool to understand and investigate the classroom behavior of children with ADHD in a controlled, systematic manner. (shrink)

This collection of new essays on sensory individuals in unimodal and multimodal perception features contributions by outstanding researchers in the fields of philosophy of perception, experimental psychology, and cognitive neuroscience. The topics investigated include conceptual, developmental, and methodological aspects of object perception, and especially how various sense modalities construct their objects from sensory features and feature bearers. The interdisciplinary approach offered has enabled new directions in research on this subject. As ordered in this volume, the topics of (...) the contributions progress from treating neural sensory processes as primarily unisensory towards approaches that take perceptual processing to be modality-independent, meta-modal, and multimodal. Even within these latter approaches, sensory stimuli, sensory properties, brain activations, and corresponding perceptual phenomenology have often been characterized in a modality-specific way. Thus, it is timely to explore the relation between those processes that are unisensory and those that are multisensory. One of the primary themes of this collection is examining whether the basic building blocks of human perception are best understood as modality-dependent units of different forms or in terms of multimodal perceptual objects. Another theme is the relation between low-level object processing (segmentation and perceptual grouping processes) and high-level object knowledge embracing object concept and object-related expectations. A final theme is the role that perceptual objects play as loci of unification in unimodal and multimodal perception, namely that they enable binding and integration of sensory properties to individual entities or events. (shrink)

BackgroundReconciling results obtained using different types of sensory measures is a challenge for autism sensory research. The present study used questionnaire, psychophysical, and neurophysiological measures to characterize autistic sensory processing in different measurement modalities.MethodsParticipants were 46 autistic and 21 typically developing 11- to 14-year-olds. Participants and their caregivers completed questionnaires regarding sensory experiences and behaviors. Auditory and somatosensory event-related potentials were recorded as part of a multisensory ERP task. Auditory detection, tactile static detection, and tactile (...) spatial resolution psychophysical thresholds were measured.ResultsSensory questionnaires strongly differentiated between autistic and typically developing individuals, while little evidence of group differences was observed in psychophysical thresholds. Crucially, the different types of measures appeared to be largely independent of one another. However, we unexpectedly found autistic participants with larger auditory Tb ERP amplitudes had reduced hearing acuity, even though all participants had hearing acuity in the non-clinical range.LimitationsThe autistic and typically developing groups were not matched on cognitive ability, although this limitation does not affect our main analyses regarding convergence of measures within autism.ConclusionOverall, based on these results, measures in different sensory modalities appear to capture distinct aspects of sensory processing in autism, with relatively limited convergence between questionnaires and laboratory-based tasks. Generally, this might reflect the reality that laboratory tasks are often carried out in controlled environments without background stimuli to compete for attention, a context which may not closely resemble the busier and more complex environments in which autistic people’s atypical sensory experiences commonly occur. Sensory questionnaires and more naturalistic laboratory tasks may be better suited to explore autistic people’s real-world sensory challenges. Further research is needed to replicate and investigate the drivers of the unexpected association we observed between auditory Tb ERP amplitudes and hearing acuity, which could represent an important confound for ERP researchers to consider in their studies. (shrink)

Sensory feedback, which can be presented in different modalities - single and combined, aids task performance in human-robot interaction (HRI). However, combining feedback modalities does not always lead to optimal performance. Indeed, it is not known how feedback modalities affect operator performance under stress. Furthermore, there is limited information on how feedback affects neural processes differently for males and females and under stress. This is a critical gap in the literature, particularly in the domain of (...) surgical robotics, where surgeons are under challenging socio-technical environments that burden them physiologically. In the present study, we posited operator performance as the summation of task performance and neurophysiological cost of maintaining that performance. In a within-subject design, we used functional near-infrared spectroscopy (fNIRS) to assess cerebral activations of 12 participants who underwent a 3D manipulation task within a virtual environment with concurrent feedback (visual and visual + haptic) in the presence and absence of a cognitive stressor. Cognitive stress was induced with the serial-7 subtraction test. We found that while task performance was higher with visual than visual + haptic feedback, it degraded under stress. The two feedback modalities were found to be associated with varying neural activities and neural efficiencies, and these were stress- and gender-dependent. Our findings engender further investigation into effectiveness of feedback modalities on males and females under stressful conditions in HRI. (shrink)

We investigate the dynamics of sensory integration for perceiving musical performance, a complex natural behavior. Thirty musically trained participants saw, heard, or both saw and heard, performances by two clarinetists. All participants used a sliding potentiometer to make continuous judgments of tension (a measure correlated with emotional response) and continuous judgments of phrasing (a measure correlated with perceived musical structure) as performances were presented. The data analysis sought to reveal relations between the sensory modalities (vision and audition) (...) and to quantify the effect of seeing the performances on participants' overall subjective experience of the music. In addition to traditional statistics, functional data analysis techniques were employed to analyze time-varying aspects of the data. The auditory and visual channels were found to convey similar experiences of phrasing but different experiences of tension through much of the performances. We found that visual information served both to augment and to reduce the experience of tension at different points in the musical piece (as revealed by functional linear modeling and functional significance testing). In addition, the musicians' movements served to extend the sense of phrasing, to cue the beginning of new phrases, to indicate musical interpretation, and to anticipate changes in emotional content. Evidence for an interaction effect suggests that there may exist an emergent quality when musical performances are both seen and heard. The investigation augments knowledge of human communicative processes spanning language and music, and involving multiple modalities of emotion and information transfer. (shrink)

Block (Trends Cogn Sci 7:285–286, 2003) and Prinz (PSYCHE 12:1–19, 2006) have defended the idea that SSD perception remains in the substituting modality (auditory or tactile). Hurley and Noë (Biol Philos 18:131–168, 2003) instead argued that after substantial training with the device, the perceptual experience that the SSD user enjoys undergoes a change, switching from tactile/auditory to visual. This debate has unfolded in something like a stalemate where, I will argue, it has become difficult to determine whether the perception acquired (...) through the coupling with an SSD remains in the substituting or the substituted modality. Within this puzzling deadlock two new approaches have been recently suggested. Ward and Meijer (Conscious Cogn 19:492–500, 2010) describe SSD perception as visual-like but characterize it as a kind of artificially induced synaesthesia. Auvray et al. (Perception 36:416–430, 2007) and Auvray and Myin (Cogn Sci 33:1036–1058, 2009) suggest that SSDs let their users experience a new kind of perception. Deroy and Auvray (forthcoming) refine this position, and argue that this new kind of perception depends on pre-existing senses without entirely aligning with any of them. So, they have talked about perceptual experience in SSDs as going "beyond vision". In a similar vein, MacPherson (Oxford University Press, New York, 2011a) claims that “if the subjects (SSD users) have experiences with both vision-like and touch-like representational characteristics then perhaps they have a sense that ordinary humans do not” (MacPherson in Oxford University Press, New York, 2011a, p. 139). (shrink)

This paper explores the challenges posed by the rapid advancement of artificial intelligence specifically Large Language Models (LLMs). I show that traditional linguistic theories and corpus studies are being outpaced by LLMs’ computational sophistication and low perplexity levels. In order to address these challenges, I suggest a focus on language as a cognitive tool shaped by embodied-environmental imperatives in the context of Agentive Cognitive Construction Grammar. To that end, I introduce an Embodied Human Language Model (EHLM), inspired by Active (...) Inference research, as a promising alternative that integrates sensory input, embodied representations, and adaptive strategies for contextualized analysis and conceptual utility maximization. By incorporating Active Inference, which sees perception as inferring the world's state from sensory data, the findings reveal that the characterization of the English modal be able to, as a triadic construction encoding biological intelligent agency, introduces a more plausible theoretical basis for the positing of linguistic constructions. This emphasizes the crucial role of embodied human language models in the comprehension of how humans construct preferred futures through language. (shrink)

Sensory Individuals: Unimodal and Multimodal Perspectives provides an interdisciplinary, well-balanced, and comprehensive look at different aspects of unisensory and multisensory objects, using both nuanced philosophical analysis and informed empirical work. The research presented in this book represents the field's progression from treating neural sensory processes as primarily modality-specific towards its current state of the art, according to which perception, and its supporting neural processes, are multi-modal, modality-independent, meta-modal, and task-dependent. Even within such approaches sensory stimuli, properties, brain (...) activations, and corresponding perceptual phenomenology can still be characterized in a modality-specific way. The book examines the basic building blocks of human perception, and whether they are best understood as sense modality dependent units of different forms or multimodal perceptual objects. The book combines a variety of innovative and integrative angles to explore the topic and acts as a catalyst for an increasingly diverse field of research, which is in an exciting phase of growth and advancement. New questions are arising as quickly as they are being answered, and the collection Sensory Individuals provides an original and up-to-date addition to the field. (shrink)

After first noting that I seek to broaden the definition of science fiction to a little more loosely defined speculative fiction, this essay explores four different ways in which fiction can work together with both the sciences and the philosophy of perception. This cooperation is needed because there is much about the sensory worlds of humans and non-human animals of which we continue to be ignorant. First, speculative fiction can be a source of hypotheses about the nature of (...) the senses. Second, it can help us understand the inner worlds of beings different from us. Third, speculative fiction often pushes us to investigate the ethical and social dimensions of sensory difference. Finally, speculative fiction can play an important role in reconciling our scientific understanding of the senses with a more commonsense understanding of same. (shrink)

Recent models of the conceptual system hold that concepts are grounded in simulations of actual experiences with instances of those concepts in sensory-motor systems (e.g., Barsalou, 1999, 2003; Solomon & Barsalou, 2001). Studies supportive of such a viewhave shown that verifying a property of a concept in one modality, and then switching to verify a property of a different concept in a different modality generates temporal processing costs similar to the cost of switching modalities in perception. In addition (...) to non-emotional concepts, the present experiment investigated switching costs in verifying properties of positive and negative (emotional) concepts. Properties of emotional concepts were taken from vision, audition, and the affective system. Parallel to switching costs in neutral concepts, the study showed that for positive and negative concepts, verifying properties from different modalities produced processing costs such that reaction times were longer and error rates were higher. Importantly, this effect was observed when switching from the affective system to sensory modalities, and vice-versa. These results support the embodied cognition view of emotion in humans. (shrink)

Here, I outline the idea of a unified hypothesis of sensory perception, developed from the theoretical vibrational mechanism of olfaction, which can be applied across all sensory modalities. I propose that all sensory perception is based upon the detection of mechanical forces at a cellular level, and the subsequent mechanotransduction of the signal via the nervous system. Thus, I argue that the sensory modalities found in the animal kingdom may all be viewed as being (...) mechanoreceptory, rather than being discrete neurophysiological systems which evolved independently of each other. I go on to argue that this idea could potentially explain language evolution, with birdsong being an example of a more simple form of non-Saussurean language that employs ‘frequency-mimicking’ to produce a vocal signal which describes acoustic, chemical and electromagnetic vibrational frequencies detected within in the environment. I also give examples of how this hypothesis could potentially explain phenomena such as vocal mimicry in animals, as well as the human perception of musicality and the occurrence of synaesthesia; a condition found in humans, where the stimulation of one sensory modality results in the stimulation of another. For example, auditory stimuli are detected and are heard as an acoustic signal, as well as being perceived as colour by the visual system. (shrink)

Essay I explores brain machine interface technologies. These make direct communication between the brain and a machine possible by means of electrical stimuli. This essay reviews the existing and emerging technologies in this field and offers an inquiry into the ethical problems that are likely to emerge. Essay II, co-written with professor Sven-Ove Hansson, presents a novel procedure to engage the public in deliberations on the potential impacts of technology. This procedure, convergence seminar, is a form of scenario-based discussion that (...) is founded on the idea of hypothetical retrospection. The theoretical background and the results of the five seminars are presented. Essay III discusses moral bioenhancement, an instance of human enhancement that alters a person’s dispositions, emotions or behavior. Moral bioenhancement could be carried out in three different ways. The first strategy is behavioral enhancement. The second strategy, favored by prominent defenders of moral enhancement, is emotional enhancement. The third strategy is the enhancement of moral dispositions, such as empathy and inequity aversion. I argue that we ought to implement a combination of the second and third strategies. Essay IV considers the possibility and potential desirability of sensory enhancement. It is proposed that existing sensory modalities in vertebrate animals are proof of concept of what is biologically possible to create in humans. Three considerations on the normative aspects of sensory enhancement are also presented in this essay. Essay V rejects disease prioritarianism, the idea that the healthcare system ought to prioritize the treatment of diseases. Instead, an approach that focuses on what medicine can accomplish is proposed. Essay VI argues that from the idea that species have an intrinsic value and that humanity has a collective responsibility to protect animal species from extinction, the conclusion that we ought to recreate species follows. Essay VII argues that unknown existential risks have not been properly addressed. It proposes a heuristic for doing so, and a concrete strategy. This strategy consists in building refuges that could withstand a large number of catastrophic events. (shrink)

It is through touch that we are able to interact directly with the world; it is our primary conduit of both pleasure and pain. Touch may be our most immediate and powerful sense—“the first sense" because of the central role it plays in experience. In this book, Matthew Fulkerson proposes that human touch, despite its functional diversity, is a single, unified sensory modality. Fulkerson offers a philosophical account of touch, reflecting the interests, methods, and approach that define contemporary (...) philosophy; but his argument is informed throughout by the insights and constraints of empirical work on touch. Human touch is a multidimensional object of investigation, Fulkerson writes, best served by using a variety of methods and approaches. -/- To defend his view of the unity of touch, Fulkerson describes and argues for a novel, unifying role for exploratory action in touch. He goes on to fill in the details of this unified, exploratory form of perception, offering philosophical accounts of tool use and distal touch, the representational structure of tangible properties, the spatial content of touch, and the role of pleasure in tactual experience. -/- Fulkerson’s argument for the unique role played by exploratory action departs notably from traditional vision-centric philosophical approaches to perception, challenging the received view that action plays the same role in all sensory modalities. The robust philosophical account of touch he offers in The First Sense has significant implications for our general understanding of perception and perceptual experience. (shrink)

Sensory dysfunction has been shown to be a part of the pathophysiology of schizophrenia. Nowadays we have an objective, non-invasive tool with which to measure neural manifestations of sensory dysfunction. Defined as time-locked changes to external stimuli in the EEG, event-related potentials (ERPs) provide an objective index of information processing in the human brain. Importantly, ERPs may be analyzed through a variety of approaches such as conventional ERP analysis, analysis in the time-frequency domain, microstate segmentation and topographical (...) analysis, as well as source localisation analysis. Each of the methods gives distinct information; they also supplement each other. Here, an attempt is made to verify the validity of combining different approaches to study sensory dysfunction in neuropsychiatric disorders. For example, the data from a schizophrenic patient and an age- and sex-matched healthy subject generate a picture of the events which emerges after visual, proprioceptive and simultaneous presentation of stimuli in both modalities. This approach, though time-consuming, allows the visualisation of changes appearing in the malfunctioning brain as compared to the healthy brain. These methods could ultimately lead to a better establishment of one or more endophenotypes for the schizophrenic disorders. They might also serve as a way to track changes in response to various medications and therapies. (shrink)

The characteristic temporal relationship between actions and their sensory outcomes allows us to distinguish self- from externally generated sensory events. However, the complex sensory environment can cause transient delays between action and outcome calling for flexible recalibration of predicted sensorimotor timing. Since the neural underpinnings of this process are largely unknown this study investigated the involvement of the cerebellum by means of cerebellar transcranial direct current stimulation. While receiving anodal, cathodal, dual-hemisphere or sham ctDCS, in an adaptation (...) phase, participants were exposed to constant delays of 150 ms between actively or passively generated button presses and visual sensory outcomes. Recalibration in the same and in another sensory modality was assessed in a subsequent test phase during which variable delays between button press and visual or auditory outcome had to be detected. Results indicated that temporal recalibration occurred in audition after anodal ctDCS while it was absent in vision. As the adaptation modality was visual, effects in audition suggest that recalibration occurred on a supra-modal level. In active conditions, anodal ctDCS improved sensorimotor recalibration at the delay level closest to the adaptation delay, suggesting a precise cerebellar-dependent temporal recalibration mechanism. In passive conditions, the facilitation of inter-sensory recalibration by anodal ctDCS was overall stronger and tuned to larger delays. These findings point to a role of the cerebellum in supra-modal temporal recalibration across sensorimotor and perceptual domains, but the differential manifestation of the effect across delay levels in active and passive conditions points to differences in the underlying mechanisms depending on the availability of action-based predictions. Furthermore, these results suggest that anodal ctDCS can be a promising tool for facilitating effects of temporal recalibration in sensorimotor and inter-sensory contexts. (shrink)

This paper attempts to answer a central question in unsupervised learning: what does it mean to “make sense” of a sensory sequence? In our formalization, making sense involves constructing a symbolic causal theory that both explains the sensory sequence and also satisfies a set of unity conditions. The unity conditions insist that the constituents of the causal theory – objects, properties, and laws – must be integrated into a coherent whole. On our account, making sense of sensory (...) input is a type of program synthesis, but it is unsupervised program synthesis. Our second contribution is a computer implementation, the Apperception Engine, that was designed to satisfy the above requirements. Our system is able to produce interpretable human-readable causal theories from very small amounts of data, because of the strong inductive bias provided by the unity conditions. A causal theory produced by our system is able to predict future sensor readings, as well as retrodict earlier readings, and impute (fill in the blanks of) missing sensory readings, in any combination. In fact, it is able to do all three tasks simultaneously. We tested the engine in a diverse variety of domains, including cellular automata, rhythms and simple nursery tunes, multi-modal binding problems, occlusion tasks, and sequence induction intelligence tests. In each domain, we test our engine's ability to predict future sensor values, retrodict earlier sensor values, and impute missing sensory data. The Apperception Engine performs well in all these domains, significantly out-performing neural net baselines. We note in particular that in the sequence induction intelligence tests, our system achieved human-level performance. This is notable because our system is not a bespoke system designed specifically to solve intelligence tests, but a general-purpose system that was designed to make sense of any sensory sequence. (shrink)

In this chapter we suggest that the origin of language, specifically the protolinguistic stage, was iconic rather than arbitrary, and fundamentally based on shared cross-modal associations. We provide evidence from natural language in the form of sound symbolism, distinguishing conventional sound symbolism from sensory sound symbolism. Sensory sound symbolism, or the presence of iconicity in natural language, is considered alongside psychological experiments in naming, and other investigations of cross-modal associations specifically involving linguistic sound. This evidence supports the idea (...) that we can directly express a variety of sensory experiences through linguistic sound, and thus that language systems have the capacity to be iconic to a large extent. We outline a theory of sensory protolanguage emergence, considering how and why arbitrariness eventually became the dominant relationship between form and meaning. Finally, we consider the possibility of continuity between cross-modal transfer in other primates and more abstract cross-modal associations among humans, suggesting this may have scaffolded the emergence of symbolic communication. (shrink)

The main stream of formal and informal logic as well as more recent work in discourse analysis provides a way of understanding certain arguments that particularly lend themselves to rational analysis. I argue, however, that these, and allied modes of analysis, be seen as heuristic models and not as the only proper mode of argument. This article introduces three other modes of argumen tation that emphasize distinct aspects of human communication, but that, at the same time, must be considered (...) for the full understanding of argumentation. These modes are (1) the emotional, which relates to the realm of feelings, (2) the visceral, which stems from the area of the physical, and (3) the kisceral, which covers the intuitive and non-sensory arenas. At its most extreme the view holds that arguments may be given (almost) wholly within one mode and not be at all susceptible to those methods of argument analysis previously used. A more cautious statement allows that any interactive argument will (possibly) contain elements from various modes, and that to attempt to reduce these all to the rational is prejudiced reductionism. (shrink)

In the last decades, the rapid growth of functional brain imaging methodologies allowed cognitive neuroscience to address open questions in philosophy and the social sciences. At the same time, novel insights from cognitive neuroscience research have begun to influence various disciplines, leading to a turn to cognition and emotion in the fields of planning and architectural design. Since 2003, the Academy of Neuroscience for Architecture has been supporting ‘neuro-architecture’ as a way to connect neuroscience and the study of behavioral responses (...) to the built environment. Among the many topics related to multisensory perceptual integration and embodiment, the concept of hapticity was recently introduced, suggesting a pivotal role of touch and haptic imagery in architectural appraisal. Arguments have thus risen in favor of the existence of shared cognitive foundations between hapticity and the supramodal functional architecture of the human brain. Precisely, supramodality refers to the functional feature of defined brain regions to process and represent specific information content in a more abstract way, independently of the sensory modality conveying such information to the brain. Here we highlight some commonalities and differences between the concepts of hapticity and supramodality according to the distinctive perspectives of architecture and cognitive neuroscience. This comparison and connection between these two different approaches may lead to novel observations in regard to people-environment relationships, and even provide empirical foundations for a renewed evidence-based design theory. (shrink)

Visuo-motor adaptation to optical prisms displacing the visual scene (prism adaptation, PA) is a method used for investigating visuomotor plasticity in healthy individuals and, in clinical settings, for the rehabilitation of unilateral spatial neglect. In the standard paradigm, the adaptation phase involves repeated pointings to visual targets, while wearing optical prisms displacing the visual scene laterally. Here we explored differences in PA, and its aftereffects (AEs), as related to the sensory modality of the target. Visual, auditory, and multisensory - (...) audio-visual - targets in the adaptation phase were used, while participants wore prisms displacing the visual field rightwards by 10◦. Proprioceptive, visual, visual-proprioceptive, auditory-proprioceptive straight-ahead shifts were measured. Pointing to auditory and to audio-visual targets in the adaptation phase produces proprioceptive, visual-proprioceptive, and auditory-proprioceptive AEs, as the typical visual targets did. This finding reveals that cross-modal plasticity effects involve both the auditory and the visual modality, and their interactions (Experiment 1). Even a shortened PA phase, requiring only 24 pointings to visual and audio-visual targets (Experiment 2), is sufficient to bring about AEs, as compared to the standard 92-pointings procedure. Finally, pointings to auditory targets cause AEs, although PA with a reduced number of pointings (24) to auditory targets brings about smaller AEs, as compared to the 92-pointings procedure (Experiment 3). Together, results from the three experiments extend to the auditory modality the sensorimotor plasticity underlying the typical AEs produced by PA to visual targets. Importantly, PA to auditory targets appears characterized by less accurate pointings and error correction, suggesting that the auditory component of the PA process may be less central to the building up of the AEs, than the sensorimotor pointing activity per se. These findings highlight both the effectiveness of a reduced number of pointings for bringing about AEs, and the possibility of inducing PA with auditory targets, which may be used as a compensatory route in patients with visual deficits. (shrink)

I review recent work from armchair and cross-cultural epistemology on whether humans possess a knowledge concept as part of a universal “folk epistemology.” The work from armchair epistemology fails because it mischaracterizes ordinary knowledge judgments. The work from cross-cultural epistemology provides some defeasible evidence for a universal folk epistemology. I argue that recent findings from comparative psychology establish that humans possess a species-typical knowledge concept. More specifically, recent work shows that knowledge attributions are a central part of primate social cognition, (...) used to predict others’ behavior and guide decision-making. The core primate knowledge concept is that of truth detection (across different sensory modalities) and retention (through memory) and may also include rudimentary forms of indirect truth discovery through inference. In virtue of their evolutionary heritage, humans inherited the primate social-cognitive system and thus share this core knowledge concept. (shrink)

The loss or absence of vision is probably one of the most incapacitating events that can befall a human being. The importance of vision for humans is also reflected in brain anatomy as approximately one third of the human brain is devoted to vision. It is therefore unsurprising that throughout history many attempts have been undertaken to develop devices aiming at substituting for a missing visual capacity. In this review, we present two concepts that have been prevalent over (...) the last two decades. The first concept is sensory substitution, which refers to the use of another sensory modality to perform a task that is normally primarily sub-served by the lost sense. The second concept is cross-modal plasticity, which occurs when loss of input in one sensory modality leads to reorganization in brain representation of other sensory modalities. Both phenomena are training-dependent. We also briefly describe the history of blindness from ancient times to modernity, and then proceed to address themeansthat have been used to help blind individuals, with an emphasis on modern technologies, invasive (various type of surgical implants) and non-invasive devices. With the advent of brain imaging, it has become possible to peer into the neural substrates of sensory substitution and highlight the magnitude of the plastic processes that lead to a rewired brain. Finally, we will address the important question of the value and practicality of the available technologies and future directions. (shrink)

Philosophers and scientists have studied sensory perception and, in particular, vision for many years. Increasingly, however, they have become interested in the nonvisual senses in greater detail and the problem of individuating the senses in a more general way. The Aristotelian view is that there are only five external senses—smell, taste, hearing, touch, and vision. This has, by many counts, been extended to include internal senses, such as balance, proprioception, and kinesthesis; pain; and potentially other human and nonhuman (...) senses. This “multisensory turn” has been driven partly by developments in contemporary psychology and neuroscience, which have revealed a host of complex interrelations and interactions between sensory modalities previously thought to be distinct. Contrasts between modalities and other crossmodal phenomena, including multisensory integration, synesthesia, and sensory substitution, have also begun to receive more attention in a burgeoning scientific and philosophical literature on multisensory perception and other crossmodal effects. This article focuses on recent empirically informed contributions to the philosophy of perception, as well as key scientific works that provide important background information and insights into the nature of the senses and sensory perception. Indeed, one of the lessons of the multisensory turn, and of contemporary philosophy of mind more generally, is that philosophers ignore this body of empirical research at their peril because many human and animal senses turn out to be richer and more complex than philosophers and scientists had previously imagined, making this a fruitful area for interdisciplinary interaction and research. (shrink)

The reach-to-grasp movement is ordinarily performed in everyday living activities and it represents a key behavior that allows humans to interact with their environment. Remarkably, it serves as an experimental test case for probing the multisensory architecture of goal-oriented actions. This review focuses on experimental evidence that enhances or modifies how we might conceptualize the “multisensory” substrates of prehension. We will review evidence suggesting that how reach-to-grasp movements are planned and executed is influenced by information coming from different sensory (...) modalities such as vision, proprioception, audition, taste, and olfaction. The review closes with some considerations about the predominant role of the multisensory constituents in shaping prehensile behavior and how this might be important for future research developments, especially in the rehabilitative domain. (shrink)

The senses, or sensory modalities, constitute the different ways we have of perceiving the world, such as seeing, hearing, touching, tasting and smelling. But how many senses are there? How many could there be? What makes the senses different? What interaction takes place between the senses? This book is a guide to thinking about these questions. Together with an extensive introduction to the topic, the book contains the key classic papers on this subject together with nine newly commissioned (...) essays.One reason that these questions are important is that we are receiving a huge influx of new information from the sciences that challenges some traditional philosophical views about the senses. This information needs to be incorporated into our view of the senses and perception. Can we do this whilst retaining our pre-existing concepts of the senses and of perception or do we need to revise our concepts? If they need to be revised, then in what way should that be done? Research in diverse areas, such as the nature of human perception, varieties of non-human animal perception, the interaction between different sensory modalities, perceptual disorders, and possible treatments for them, calls into question the platitude that there are five senses, as well as the pre-supposition that we know what we are counting when we count them as five.This book will serve as an inspiring introduction to the topic and as a basis from which further new research will grow. (shrink)

Is consciousness multisensory? Obviously it is multisensory in certain ways. Human beings typically possess the capacity to have experiences in at least the five familiar sensory modalities, and quite possibly in a number of other less commonly recognised modalities as well. But there are other respects in which it is far from obvious that consciousness is multisensory. This chapter is concerned with one such respect. Οur concern here is with whether consciousness contains experiences associated with distinct (...) modalities at the same time. We will describe those who endorse a positive answer to this question as endorsing a multisensory view (MSV) of the structure of consciousness, and those who endorse a negative answer to this question as having a unisensory view (USV) of the structure of consciousness. (shrink)

Are attributions of content and function determinate, or is there no fact of the matter to be fixed? Daniel Dennett has argued in favor of indeterminacy and concludes that, in practice, content and function cannot be fixed. The discovery of an electrical modality in vertebrates offers one concrete instance where attributions of function and content are supported by a strong scientific consensus. A century ago, electroreception was unimagined, whereas today it is widely believed that many species of bony fish, amphibians, (...) sharks, skates, and rays possess this non-human sensory modality. A look at the history of science related to this discovery reveals a highly interdisciplinary endeavor, encompassing ethology, behavioral analysis, neuroscience, and evolutionary biology. While each area provides important evidence, none is sufficient on its own to fix content and function. Instead, I argue that an interdisciplinary, neuroethological approach is required to carry out such determinations. Further, a detailed consideration of biological research suggests that while content and function claims are empirically underdetermined and uncertain, there is insufficient reason to believe in an additional problem of indeterminism. In particular, Dennett's indeterminism arises from a research methodology -- logical adaptationism -- that generates evidence from only one of the areas of neuroethology. However, logical adaptationism does not reflect adaptationism as it is practiced in contemporary biology. I conclude that Dennett is faced with a dilemma: On the one hand, he can hold to logical adaptationism and the indeterminism that results from it, while giving up the relevance of his arguments to biological practice. On the other, he can embrace a more accurate version of adaptationism -- one which plays a role in a larger neuroethological framework -- but from which no strong indeterminacy claims follow. (shrink)

Numerous studies have shown that stimulus-response-compatibility (SRC) effects in the go-nogo version of the Simon task can be elicited as a result of performing the task together with another human or non-human agent (e.g., a Japanese-waving-cat, a working-clock, or a ticking-metronome). A parsimonious explanation for both social and non-social SRC effects is that highlighting the spatial significance of alternative (non-/social) action events makes action selection more difficult. This holds even when action events are task-irrelevant. Recent findings, however, suggest (...) that this explanation holds only for cases of a modality correspondence between the Simon task as such (i.e., auditory or visual) and the alternative (non-/social) action event that needs to be discriminated. However, based on the fact that perception and action are represented by the same kind of codes, an event that makes the go-nogo decision more challenging should impact go-nogo Simon task performance. To tackle this issue, the present study tested if alternative stimulus events that come from a different sensory modality do impact SRC effects in the go-nogo version of the Simon task. This was tested in the presence and absence of alternative action events of a human co-actor. In a multimodal (auditory–visual) go-nogo Simon paradigm, participants responded to their assigned stimulus – e.g., a single auditory stimulus while ignoring the alternative visual stimulus or vice versa – in the presence or absence of a human co-actor (i.e., joint and single go-nogo condition). Results showed reliable SRCs in both, single and joint go-nogo Simon task conditions independent of the modality participants had to respond to. Although a correspondence between stimulus material and attention-grabbing event might be an efficient condition for SRCs to emerge, the driving force underlying the emergence of SRCs rather appears to be whether the attentional focus prevents or facilitates alternative events to be integrated. Thus, under task conditions in which the attentional focus is sufficiently broad to enable the integration and thus cognitive representation of alternative events, go-nogo decisions become more difficult, resulting in reliable SRCs in single and joint go-nogo Simon tasks. (shrink)

The integration of multisensory information is an essential mechanism in perception and action control. Research in multisensory integration is concerned with how the information from the different sensory modalities, such as the senses of vision, hearing, smell, taste, touch, and proprioception, are integrated to a coherent representation of objects (for an overview, see e.g., Calvert, Spence and Stein, 2004). The combination of information from the different senses is central for action control. For instance, when you grasp for a (...) rubber duck, you can see its size, feel its compliance and hear the sound it produces. Moreover, identical physical properties of an object can be provided by different senses. You can both see and feel the size of the rubber duck. Even when you grasp for the rubber duck with a tool (e.g., with tongs), the information from the proximal hand, from the effective part of the distal tool and from the eyes are integrated in a manner to act successfully (for limitations of this integration see Sutter et al., 2013).Over the recent decade a surge of interest in multisensory integration and action control has been witnessed, especially in connection with the idea of a statistically optimized integration of multiple sensory sources. The human information processing system is assumed to adjust moment-by-moment the relative contribution of each sense’s estimate to a multisensory task. The sense’s contribution depends on its variance, so that the total varia... (shrink)

Miracles are extraordinary events that occur in the hands of those who claim to be prophets and which cannot be repeated by others. By these miracles, the prophets prove to society that the truths they convey are of divine origin. The miracles bestowed upon prophets vary according to the scope of the message they deliver and the interests and relevance of the societies with which they deal. Accordingly, Islamic scholars have classified miracles into three groups. The miraculous events that appeal (...) to people’s senses and lead them to the truths they convey by captivating their astonishment are called emotional miracles. Such miracles are events that occur without cause by overcoming the laws of nature, which are in the realm of causes, through the direct intervention of Allāh. On the other hand, spiritual miracles are truths that arouse the interlocutor’s interest in moral conduct and rational conclusions rather than in his sensory perceptions. Since these truths are attained through contemplation, they are not confined to a particular time and place, as in the case of emotional miracles. Some Kalām scholars have also considered the Prophets’ revelational knowledge about events that have occurred in the past or will occur in the future as miracles. They have studied these events that a person could not bear in the past or could not predict for the future under the title of "The news miracle" Among these three types of miracles, scholars have considered spiritual miracles superior to the others both in terms of the breadth of their influence and their continuity. That Prophet Muhammad (PBUH) is the last of the prophets, and that the teachings he received through revelation remain valid until the Day of Judgment necessitates that some of the miracles bestowed upon him must be of a quality that attracts the attention of all people. For this reason, Islamic scholars have placed the permanent miracle of Prophet Muhammad in the category of spiritual miracles. Scholars agree that his greatest miracle that has astonished all humankind is the Qurʾān. The clearest proof of this are the verses in the Qurʾān that challenge the opponents to bring a similar one. Allāh commands the disbelievers first to bring a similitude of the Qurʾān (al-Isrāʾ 17:88), then the equivalent of ten surahs (Hūd 11:13), and finally any surah of the Qurʾān (al-Baqara 2:23). Thus, the deniers are made aware of their inability to meet this challenge under gradually mitigating circumstances. Although scholars agree on the Qurʾānic miracle’s reality, they have expressed different views on its nature and essence. Thus, some scholars have argued that the miracle of the Qurʾān does not arise from some properties special to it but from the direct intervention of Allāh in human who have the ability to bring something like it. Most Islamic scholars, on the other hand, pointed out that the Qurʾān has aspects that exceed human powers in various fields, and they explained the inability to reply in the same vein to the Qurʾān due to its uniquenesses. Accordingly, some scholars have argued that miracles are perceptible, but it is never possible to define them. Other scholars, however, viewed miracles as an identifiable phenomenon and stated that the Qurʾān is an unusual and unique book in terms of verses. In this article, the questions Why was the Quran sent down as a miraculous Kalam with an attribute not found in other divine books and in what way did the Qurʾān make people weak to have a similar of it? were scrutinized, thus, some fundamental questions regarding the nature and character of iʿdjāz al-Qurʾān are elucidated. Since the topic deals with history and literature, scanning sources and data analysis were mainly used as methods in the research. The results can be summerized as follows: Although the Prophet was supported by both emotional and mental miracles in proving the Prophetic case, his greatest miracle is the Quran. The miracle of the Qurʾān is different from the other miracles, based on both the challenging verses and the Prophet's explanations of the miracles. The reason for this difference lies in the fact that the miracle of the Qurʾān is based on reason, unlike the other miracles which are emotional. Most Islamic scholars who deal with the miracle of the Qurʾān emphasize the issues of rhetoric, verse, style, and content. At this point, they emphasized that the Qurʾān has a unique expression and content that cannot be surpassed. (shrink)

Language’s intentional nature has been highlighted as a crucial feature distinguishing it from other communication systems. Specifically, language is often thought to depend on highly structured intentional action and mutual mindreading by a communicator and recipient. Whilst similar abilities in animals can shed light on the evolution of intentionality, they remain challenging to detect unambiguously. We revisit animal intentional communication and suggest that progress in identifying analogous capacities has been complicated by (i) the assumption that intentional (that is, voluntary) production (...) of communicative acts requires mental-state attribution, and (ii) variation in approaches investigating communication across sensory modalities. To move forward, we argue that a framework fusing research across modalities and species is required. We structure intentional communication into a series of requirements, each of which can be operationalised, investigated empirically, and must be met for purposive, intentionally communicative acts to be demonstrated. Our unified approach helps elucidate the distribution of animal intentional communication and subsequently serves to clarify what is meant by attributions of intentional communication in animals and humans. (shrink)

In human conscious experience, many features are present in combination: objects are presented through the senses, information from different sensory modalities is integrated, events are marked wit...

Why is there a 'hard problem' of consciousness? Why do we seem unable to grasp intuitively that physical brain processes can be identical to experiences? Here I comment on the 'meta-problem' (Chalmers, 2018), based on previous ideas (Storm, 2014; 2018). In short: humans may be 'inborn dualists' ('neuroscepticism'), because evolution gave us two (types of) brain systems (or functional modes): one (Sp) for understanding relatively simple physical phenomena, and another (Sm) specialized for mental phenomena. Because Sp cannot deal with the (...) immense complexity of the brain processes underlying consciousness, it represents them as fundamentally different from nonmental physical phenomena (dualist intuition), using 'simulations' to produce 'Sm-type understanding'/predictions that seems radically different from 'Sp-type understanding'. (By analogy, different sensory modalities, handled by distinct brain systems, evoke qualitatively different experiences.) Brain systems for Sp representations of our brain processes never evolved, because they would be useless. When lacking a single 'template' matching different aspects of reality (objective vs. subjective = simulated), complementary 'models' are needed ('neuro-complementarity'), like the wave–particle duality in quantum mechanics. Thus, it seems plausible that Sp and Sm evolved because they were needed to cope with different challenges, and that 'problem intuitions' are side effects of these useful but different brain systems. (shrink)

In this paper I contend that abduction is essentially multimodal, in that both data and hypotheses can have a full range of verbal and sensory representations, involving words, sights, images, smells, etc. but also kinesthetic experiences and other feelings such as pain, and thus all sensory modalities. The kinesthetic aspects simply explain abductive reasoning is basically manipulative, both linguistic and non linguistic signs have an internal semiotic life, as particular configurations of neural networks and chemical distributions (and (...) in terms of their transformations) at the level of human brains, and as somatic expressions, but can also be delegated to many external objects and devices, for example written texts, diagrams, artifacts, etc. Still in this case unconscious aspects takepart of the abductive procedure, which acquires the features of a kind of "thinking through doing". (shrink)

This paper proposes a dynamic theory of embodiment that aims to get beyond the absent moving body in embodied social theory. The first somatic revolution, inspired by Merleau Ponty, provided theories based on the feeling and experience of the body. The theory of dynamic embodiment focuses instead on the doing itself as embodied social action, in which the embodied person is fore-grounded as a complex resource for meaning making. This represents a theoretical enrichment of the earlier turn to the body (...) in social theory, which tended to separate the semiotic, as necessarily representational and/or linguistic, from the somatic as a wide range of corporeal processes and practices assumed to be separated from mind, language and/or conscious thought. We argue that overcoming this persistent Cartesianism requires a New Realist approach to the proper location of human agency as a causal power, one that promotes a bio-psycho-social concept of personhood. Part one of the paper presents a general framework for this perspective, while part two applies this paradigm ethnographically to illustrate how bringing semiosis and somatics together requires a robust conception of multi-sensory modalities. (shrink)

Geometrical intuitions spontaneously drive visuo-spatial reasoning in human adults, children and animals. Is their emergence intrinsically linked to visual experience, or does it reflect a core property of cognition shared across sensory modalities? To address this question, we tested the sensitivity of blind-from-birth adults to geometrical-invariants using a haptic deviant-figure detection task. Blind participants spontaneously used many geometric concepts such as parallelism, right angles and geometrical shapes to detect intruders in haptic displays, but experienced difficulties with symmetry (...) and complex spatial transformations. Across items, their performance was highly correlated with that of sighted adults performing the same task in touch (blindfolded) and in vision, as well as with the performances of uneducated preschoolers and Amazonian adults. Our results support the existence of an amodal core-system of geometry that arises independently of visual experience. However, performance at selecting geometric intruders was generally higher in the visual compared to the haptic modality, suggesting that sensory-specific spatial experience may play a role in refining the properties of this core-system of geometry. (shrink)

Vision is the main entrance for environmental input to the human brain. Even if vision is our most used sensory modality, its importance is not limited to environmental exploration. Rather it has strong links to motor competences, further extending to cognitive and social aspects of human life. These multifaceted relationships are particularly important in developmental age and become dramatically evident in presence of complex deficits originating from visual aberrancies. The present review summarizes the available neuropsychological evidence on (...) the development of visual competences, with a particular focus on the associated visuo-motor integration skills in health and disease. With the aim of supporting future research and interventional settings, the goal of the present review is to constitute a solid base to help the translation of neuropsychological hypotheses into straightforward empirical investigations and rehabilitation/training protocols. This approach will further increase the impact, ameliorate the acceptance, and ease the use and implementation of lab-derived intervention protocols in real-life situations. (shrink)

Intermodal representationalists hold that the phenomenal characters of experiences are fully determined by their contents. In contrast, intramodal representationalists hold that the phenomenal characters of experiences are determined by their contents together with their intentional modes or manners of representation, which are nonrepresentational features corresponding roughly to the sensory modalities. This paper discusses a kind of experience that provides evidence for an intermodal representationalist view: intermodal experiences, experiences that unify experiences in different modalities. I argue that such (...) experiences are much easier to explain on the intermodal view. (shrink)

In this study bibliometric mapping method was employed to visualise the current research trends and the impact of the two most influential models of working memory, namely: A. D. Baddeley and G. J. Hitch’s multicomponent working memory model and N. Cowan’s embedded-processes model of working memory. Using VOSviewer software two maps were generated based on the index-term words extracted from the research papers citing Baddeley and Cowan, respectively. The maps represent networks of co-occurrences of index terms and can be interpreted (...) as an indication of the main research fields related to the examined models of WM. The results of the analysis revealed that the spheres of influence of the two main conceptualisations of WM are rather different than similar. Although the first two clusters, i.e. “brain mapping” and “higher-level cognition and development” are present in both maps, their relative importance varies. The remaining clusters are unique to each map. Baddeley’s theory seems to have a greater influence on “neuropsychology”, while Cowan’s theory - on basic research on “biological systems”, including the nervous system in humans and animals. The second difference between these theories concerns their relations to functions and dysfunctions associated with particular sensory modalities: in Baddelay’s theory with the “auditory modality” cluster, and in Cowan’s - with the “visual modality” one. (shrink)

This article introduces the notion of social synaesthesia as a means of theorizing about mindbody—society relationships. Following a brief discussion of synaesthesia, the concept of social synaesthesia is developed. This concept is then illustrated using photic (light) experiences among devotees in a religious group. It is argued that these experiences illustrate the materialization or embodiment of the experience of charisma in a leader’s presence. It is also suggested that such experiences are more intense and vivid forms of the quotidian tendency (...) to ‘make the word flesh’ — that is, express cognitive-emotional experiences in a physical idiom. Bioagency (as a locus of embodied charisma) is then discussed as the process through which the ‘word is made flesh’. The conclusion argues that just as synaesthetic experiences point to an interpenetration and melding of sensory modalities and challenge mindbody dualism, social synaesthesia calls into question mindbody/social dualisms. Finally, such experiences are seen as more vivid and intense forms of ordinary human capacities. (shrink)

Proprioception is a crucial sensory modality involved in the control and regulation of coordinated movements and in motor learning. However, the extent to which proprioceptive acuity is influenced by local muscle fatigue is obscured by methodological differences in proprioceptive and fatiguing protocols. In this study, we used high resolution kinematic measurements provided by a robotic device, as well as both frequency and time domain analysis of signals captured via surface electromyography (sEMG) to examine the effects of local muscle fatigue (...) on wrist proprioceptive acuity in 16 physically and neurologically healthy young adults. To this end, participants performed a flexion/extension ipsilateral joint position matching test (JPM), after which a high-resistive robotic task was used to induce muscle fatigue of the flexor carpi radialis (FCR) muscle. The JPM test was then repeated in order to analyze potential changes in proprioceptive acuity. Results indicated that the fatigue protocol had a significant effect on movements performed in flexion direction, with participants exhibiting a tendency to undershoot the target before the fatigue protocol (-1.218°), but overshooting after the fatigue protocol (0.587°). In contrast, in the extension direction error bias values were similar before and after the fatigue protocol as expected (pre = -1.852°, post = -1.237°) and reflected a tendency to undershoot the target. Moreover, statistical analysis indicated that movement variability was not influenced by the fatigue protocol or movement direction. In sum, results of the present study demonstrate that an individual’s estimation of wrist joint displacement (i.e., error bias), but not precision (i.e., variability), is affected by muscular fatigue in a sample of neurologically and physically healthy adults. (shrink)

Is there a space of the sensory modalities? Such a space would be one in which we can represent all the actual, and at least some of the possible, sensory modalities. The relative position of the senses in this space would indicate how similar and how different the senses were from each other. The construction of such a space might reveal unconsidered features of the actual and possible senses, help us to define what a sense is, (...) and provide grounds that we might use to decide what is one token sense rather than multiple token senses. -/- In this paper, I explore, refine, and defend the idea that we can construct such a space—an idea that I briefly proposed in earlier work (Macpherson 2011a and 2011b). In doing so, I defend the idea from an objection that Richard Gray (2012) has voiced. Gray claims that the dimensions of the space of sensory modalities must generate a non-­‐arbitrary ordering of the senses. He argues that we cannot find dimensions that do so. I disagree. I identify different ways in which a space could be arbitrary but I argue that we can define a space of the sensory modalities in a non-arbitrary manner. I give examples of what the dimensions might be. (shrink)

Some sensory modalities, such as sight, touch and audition, are arguably spatial, and one way to understand these spatial senses is to investigate spatial representations in them. Here I focus on a specific element in this area— the interplay between perspectival variation and spatial constancy—and discuss recent interdisciplinary works on this topic. With these relevant experimental works, we will see clearly how traditional controversies in philosophy, for example, whether we perceive perspectival shapes as well as objective shapes, and (...) whether any sensory field should be postulated in senses other than vision, can benefit from considerations in empirical sciences. (shrink)