Intermodal representationalists hold that the phenomenal characters of experiences are fully determined by their contents. In contrast, intramodal representationalists hold that the phenomenal characters of experiences are determined by their contents together with their intentional modes or manners of representation, which are nonrepresentational features corresponding roughly to the sensory modalities. This paper discusses a kind of experience that provides evidence for an intermodal representationalist view: intermodal experiences, experiences that unify experiences in different modalities. I argue that such (...) experiences are much easier to explain on the intermodal view. (shrink)

Some sensory modalities, such as sight, touch and audition, are arguably spatial, and one way to understand these spatial senses is to investigate spatial representations in them. Here I focus on a specific element in this area— the interplay between perspectival variation and spatial constancy—and discuss recent interdisciplinary works on this topic. With these relevant experimental works, we will see clearly how traditional controversies in philosophy, for example, whether we perceive perspectival shapes as well as objective shapes, and (...) whether any sensory field should be postulated in senses other than vision, can benefit from considerations in empirical sciences. (shrink)

Is there a space of the sensory modalities? Such a space would be one in which we can represent all the actual, and at least some of the possible, sensory modalities. The relative position of the senses in this space would indicate how similar and how different the senses were from each other. The construction of such a space might reveal unconsidered features of the actual and possible senses, help us to define what a sense is, (...) and provide grounds that we might use to decide what is one token sense rather than multiple token senses. -/- In this paper, I explore, refine, and defend the idea that we can construct such a space—an idea that I briefly proposed in earlier work (Macpherson 2011a and 2011b). In doing so, I defend the idea from an objection that Richard Gray (2012) has voiced. Gray claims that the dimensions of the space of sensory modalities must generate a non-­‐arbitrary ordering of the senses. He argues that we cannot find dimensions that do so. I disagree. I identify different ways in which a space could be arbitrary but I argue that we can define a space of the sensory modalities in a non-arbitrary manner. I give examples of what the dimensions might be. (shrink)

Is the flavor of mint reducible to the minty smell, the taste, and the menthol-like coolness on the roof of one’s mouth, or does it include something over and above these—something not properly associated with any one of the contributing senses? More generally, are there features of perceptual experiences—so-called novel features—that are not associated with any of our senses taken singly? This question has received a lot of attention of late. Yet surprisingly little attention has been paid to the question (...) of what it means to say that a feature is associated with a sensory modality in the first place. Indeed, there is only one fully developed proposal in the literature, due to Casey O’Callaghan. I argue that this proposal is too permissive to inform the debate over novel features. I go on to argue that all attempts to formulate a better proposal along these lines fail. The corollary of my arguments is that the question of the existence of novel features is poorly formed. Furthermore, the problem generalizes, with the result that we should not rely on our pre-theoretical notions of the senses as the basis of theorizing about the features of perceptual experiences. (shrink)

Perceptual experience strikes us as a presentation of the here and now. I argue that it also involves experience of the past. This claim is often made based on cases, like seeing stars, involving significant signal-transmission lag, or based on how working memory allows us to experience extended events. I argue that the past is injected into perceptual experience via a third way: long-term memory traces in sensory circuits. Memory, like the receptor-based senses, is an integrated and constituent modality (...) through which we experience the environment. Because of this modality, we experience the sensed properties of stimuli partly as they are now, but also partly as we encountered them in the past. (shrink)

Pain is commonly explained in terms of the perceptual activity of a distinct sensory modality, the function of which is to enable us to perceive actual or potential damage to the body. However, the characterization of pain experience in terms of a distinct sensory modality with such content is problematic. I argue that pain is better explained as occupying a different role in relation to perception: to indicate when the stimuli that are sensed in perceiving anything by means (...) of a sensory modality exceed a significant level of intensity. Viewing the system underlying pain experience as an integral and functionally integrated feature of all the senses provides a new perspective on the diverse forms of pain, what it is like to experience pain and the contrasting natures of pain and pleasure. (shrink)

Perception can provide us with a privileged source of evidence about the external world – evidence that makes it rational to believe things about the world. In Reasons First, Mark Schroeder offers a new view on how perception does so. The central motivation behind Schroeder’s account is to offer an answer to what evidence perception equips us with according to which it is what he calls world-implicating but non-factive, and thereby to glean some of the key advantages of both externalism (...) and internalism, respectively. He answers this motivation by developing a more specific view that he calls the Apparent Factive Attitude view, which pairs an answer to what evidence is provided by a perceptual experience with an answer to why having that perceptual experience provides you with that evidence. In this paper we advance two interconnected problems for Schroeder’s Apparent Factive Attitude view. A traditional intuitive judgment that often motivates internalists is the idea that internal duplicates must necessarily be equally rational in whatever beliefs they have. Schroeder’s arguments rely on a weaker claim - that people who are both internal and historical external duplicates but differ only in the veridicality of a single perceptual experience must be equally rational in whatever beliefs they have. In this way he preserves what he argues to be a more compelling internalist intuition. But our arguments will show that Schroeder’s view is committed to denying an even more compelling internalist intuition yet - that internal duplicates must have the same phenomenology. (shrink)

This paper aims to analyze the concepts of phenomenal experiences, subjective standpoints, and self-awareness in light of an enactive approach to mind. In doing so, it assumes that a clear-cut reorganization of those three concepts will help to deal more safely with technological enhancements of sensory modalities, including the hypothetical introspection of a person’s own neural states. It argues in favor of understanding the mental as a subcategory of the physical and of considering measurement scales and mutually complementary (...) perspectives while tackling the mind-body relation. After discussing realistic prospects of sensory enhancement, it explores the overlaps of the notions of phenomenal experiences, subjective standpoints, and self-awareness, concluding for a reorganization that seems convenient to understand new neurotechnologies more effectively. (shrink)

Results from studies on gender differences in emotion recognition vary, depending on the types of emotion and the sensory modalities used for stimulus presentation. This makes comparability between different studies problematic. This study investigated emotion recognition of healthy participants (N = 84; 40 males; ages 20 to 70 years), using dynamic stimuli, displayed by two genders in three different sensory modalities (auditory, visual, audio-visual) and five emotional categories. The participants were asked to categorise the stimuli on (...) the basis of their nonverbal emotional content (happy, alluring, neutral, angry, and disgusted). Hit rates and category selection biases were analysed. Women were found to be more accurate in recognition of emotional prosody. This effect was partially mediated by hearing loss for the frequency of 8,000 Hz. Moreover, there was a gender-specific selection bias for alluring stimuli: Men, as compared to women, chose “alluring” more often when a stimulus was presented by a woman as compared to a man. (shrink)

Joint attention is typically conceptualized as a robust psychological phenomenon. In philosophy, this apparently innocuous assumption leads to the problem of accounting for the “openness” of joint attention. In psychology, it leads to the problem of justifying alternative operationalizations of joint attention, since there does not seem to be much which is psychologically uniform across different joint attentional engagements. Contrary to the received wisdom, I argue that joint attention is a social relationship which normatively regulates the attentional states of two (...) or more individuals. This social account of joint attention leans on Bart Geurts’ view of communication as commitment sharing. Its promises are: (i) to explain the role of joint attention in wider joint activities, including communicative interactions; (ii) to account for how playing this role requires individuals to deploy different psychological resources on different occasions; and (iii) to identify the rationale behind alternative operationalizations of joint attention. (shrink)

A key question in studying consciousness is how neural operations in the brain can identify streams of sensory input as belonging to distinct modalities, which contributes to the representation of qualitatively different experiences. The basis for identification of modalities is proposed to be constituted by self-organized comparative operations across a network of unimodal and multimodal sensory areas. However, such network interactions alone cannot answer the question how sensory feature detectors collectively account for an integrated, yet (...) phenomenally differentiated experiential content. This problem turns out to be different from, although related to, the binding problem. It is proposed that the neural correlate of an enriched, multimodal experience is constituted by the attractor state of a dynamic associative network. Within this network, unimodal and multimodal sensory maps continuously interact to influence each other’s attractor state, so that a feature change in one modality results in a fast re-coding of feature information in another modality. In this scheme, feature detection is coded by firing-rate, whereas firing phase codes relational aspects. (shrink)

BackgroundAttention deficit hyperactivity disorder is clinically diagnosed; however, quantitative analysis to statistically analyze the symptom severity of children with ADHD via the measurement of head movement is still in progress. Studies focusing on the cues that may influence the attention of children with ADHD in classroom settings, where children spend a considerable amount of time, are relatively scarce. Virtual reality allows real-life simulation of classroom environments and thus provides an opportunity to test a range of theories in a naturalistic and (...) controlled manner. The objective of this study was to investigate the correlation between participants’ head movements and their reports of inattention and hyperactivity, and to investigate how their head movements are affected by different social cues of different sensory modalities.MethodsThirty-seven children and adolescents with and without ADHD were recruited for this study. All participants were assessed for diagnoses, clinical symptoms, and self-reported symptoms. A virtual reality-continuous performance test was conducted under four conditions: control, no-cue, visual cue, and visual/audio cue. A quantitativecomparison of the participants’ head movements was conducted in three dimensions using a head-mounted display in a VR classroom environment. Task-irrelevant head movements were analyzed separately, considering the dimension of movement needed to perform the VR-CPT.ResultsThe magnitude of head movement, especially task-irrelevant head movement, significantly correlated with the current standard of clinical assessment in the ADHD group. Regarding the four conditions, head movement showed changes according to the complexity of social cues in both the ADHD and healthy control groups.ConclusionChildren and adolescents with ADHD showed decreasing task-irrelevant movements in the presence of social stimuli toward the intended orientation. As a proof-of-concept study, this study preliminarily identifies the potential of VR as a tool to understand and investigate the classroom behavior of children with ADHD in a controlled, systematic manner. (shrink)

Two proposals have recently, and independently, been made about a space of possible sensory modalities. In this paper I examine these different proposals, and offer one of my own. I suggest that there are several spaces associated with distinct kinds of sensory modality.

Research on sequence effects on learning visual categories has shown that interleaving (i.e., studying the categories in a mixed manner) facilitates category induction as compared to blocking (i.e., studying the categories one by one), but learners are unaware of the interleaving effect and prefer blocking. However, little attention has been paid to sequence effects in perceptual learning across further sensory modalities. The present (preregistered) research addresses this shortcoming by using auditory (birdcalls), olfactory (tealeaves), gustatory (ingredient mixtures), and tactile (...) (stones) stimuli across four analog experiments. The number of participants per experiment was determined based on a medium effect size of interleaving. Participants studied six categories (with six exemplars, respectively) either interleaved or blocked. No single experiment showed a significant effect of interleaving. We ran a comprehensive meta‐analysis based on the data from all experiments, which revealed a significant small effect of interleaving, demonstrating its applicability to perceptual learning across all sensory modalities. Learners in the interleaved condition underestimated their classification performance. Overall, learners did not rate interleaving as less effective than blocking, which is at odds with previous studies that consistently demonstrated a metacognitive preference of blocking. Our findings suggest that learners rely less on conventional beliefs about the effectivity of study sequence when dealing with unfamiliar (blindfolded) perceptual learning tasks. (shrink)

This collection of new essays on sensory individuals in unimodal and multimodal perception features contributions by outstanding researchers in the fields of philosophy of perception, experimental psychology, and cognitive neuroscience. The topics investigated include conceptual, developmental, and methodological aspects of object perception, and especially how various sense modalities construct their objects from sensory features and feature bearers. The interdisciplinary approach offered has enabled new directions in research on this subject. As ordered in this volume, the topics of (...) the contributions progress from treating neural sensory processes as primarily unisensory towards approaches that take perceptual processing to be modality-independent, meta-modal, and multimodal. Even within these latter approaches, sensory stimuli, sensory properties, brain activations, and corresponding perceptual phenomenology have often been characterized in a modality-specific way. Thus, it is timely to explore the relation between those processes that are unisensory and those that are multisensory. One of the primary themes of this collection is examining whether the basic building blocks of human perception are best understood as modality-dependent units of different forms or in terms of multimodal perceptual objects. Another theme is the relation between low-level object processing (segmentation and perceptual grouping processes) and high-level object knowledge embracing object concept and object-related expectations. A final theme is the role that perceptual objects play as loci of unification in unimodal and multimodal perception, namely that they enable binding and integration of sensory properties to individual entities or events. (shrink)

Many philosophers believe that there is a causal condition on perception, and that this condition is a conceptual truth about perception. A highly influential argument for this claim is based on intuitive responses to Gricean-style thought experiments. Do the folk share the intuitions of philosophers? Roberts et al. (2016) presented participants with two kinds of cases: Blocker cases (similar to Grice’s case involving a mirror and a pillar) and Non-Blocker cases (similar to Grice’s case involving a clock and brain stimulation). (...) They found that a substantial minority agreed that seeing occurs in the Non-Blocker cases, and that in the Blocker cases significantly less agreed that seeing occurs. They thus hypothesized that folk intuitions better align with a no blocker condition than with a causal condition. This paper continues this line of enquiry with two new experiments. The paper investigates the generality and robustness of Roberts et al.’s findings by expanding the sense modalities tested from only vision to audition and olfaction as well. The paper also uses Gricean-style thought experiments as a case study for investigating the “reflection defense” against the negative project in experimental philosophy. Our results replicate and extend Roberts et al.’s study and support their hypothesis that folk intuitions better align with a no blocker condition. They also provide an empirical reason to doubt the reflection defense. (shrink)

Many philosophers believe that there is a causal condition on perception, and that this condition is a conceptual truth about perception. A highly influential argument for this claim is based on intuitive responses to Gricean style thought experiments. Do the folk share the intuitions of philosophers? Roberts et al. (2016) presented participants with two kinds of cases: Blocker cases (similar to Grice’s case involving a mirror and a pillar) and Non-Blocker cases (similar to Grice’s case involving a clock and brain (...) stimulation). They found that a substantial minority agreed that seeing occurs in the Non-Blocker cases, and that in the Blocker cases significantly less agreed that seeing occurs. They thus hypothesized that folk intuitions better align with a no blocker condition than with a causal condition. This paper continues this line of enquiry with two new experiments. The paper investigates the generality and robustness of Roberts et al.’s findings by expanding the sense modalities tested from only vision to audition and olfaction as well. The paper also uses Gricean style thought experiments as a case study for investigating the “reflection defense” against the negative project in experimental philosophy. Our results replicate and extend Roberts et al.’s study and support their hypothesis that folk intuitions better align with a no blocker condition. They also provide an empirical reason to doubt the reflection defense. (shrink)

BackgroundReconciling results obtained using different types of sensory measures is a challenge for autism sensory research. The present study used questionnaire, psychophysical, and neurophysiological measures to characterize autistic sensory processing in different measurement modalities.MethodsParticipants were 46 autistic and 21 typically developing 11- to 14-year-olds. Participants and their caregivers completed questionnaires regarding sensory experiences and behaviors. Auditory and somatosensory event-related potentials were recorded as part of a multisensory ERP task. Auditory detection, tactile static detection, and tactile (...) spatial resolution psychophysical thresholds were measured.ResultsSensory questionnaires strongly differentiated between autistic and typically developing individuals, while little evidence of group differences was observed in psychophysical thresholds. Crucially, the different types of measures appeared to be largely independent of one another. However, we unexpectedly found autistic participants with larger auditory Tb ERP amplitudes had reduced hearing acuity, even though all participants had hearing acuity in the non-clinical range.LimitationsThe autistic and typically developing groups were not matched on cognitive ability, although this limitation does not affect our main analyses regarding convergence of measures within autism.ConclusionOverall, based on these results, measures in different sensory modalities appear to capture distinct aspects of sensory processing in autism, with relatively limited convergence between questionnaires and laboratory-based tasks. Generally, this might reflect the reality that laboratory tasks are often carried out in controlled environments without background stimuli to compete for attention, a context which may not closely resemble the busier and more complex environments in which autistic people’s atypical sensory experiences commonly occur. Sensory questionnaires and more naturalistic laboratory tasks may be better suited to explore autistic people’s real-world sensory challenges. Further research is needed to replicate and investigate the drivers of the unexpected association we observed between auditory Tb ERP amplitudes and hearing acuity, which could represent an important confound for ERP researchers to consider in their studies. (shrink)

In the present study, cross-modal (i.e., across sensory modalities such as smell and sound) iconicity (i.e., resemblance) and indexicality (i.e., contiguity) in lexical sensory and emotional signs in Finnish Sign Language will be considered from an articulatory perspective (i.e., the production of signs). Such cross-modal iconicity has not been extensively studied previously, so here, with the help of cognitive semiotics, I aim to carefully describe the cross-modal patterns observed across 118 signs, including 60 sensory signs and (...) 58 emotional signs. The analysis is framed within the theoretical model of Semiotic Hierarchy, which entails a non-reductionist view of meaning. In addition, a pheno-methodological triangulation will be applied: phenomenology (first-person method), literature of phenomenological and semiotic descriptions (second-person perspective) and experimental findings (third-person perspective). The results of this analysis show that (a) 71 of the 118 sensory and emotional signs are cross-modally indexical, (b) only 10 of the 71 signs can be regarded as cross-modally iconic, (c) cross-modal iconicity is highly diagrammatic, (d) iconicity and indexicality are highly integrated, and (e) articulatory feedback matters in the formation of semiotic patterns. This study contributes to our understanding of cross-modal iconicity in signed languages, as well as studies in semiotic systems more generally. (shrink)

Sensory substitution devices provide through an unusual sensory modality (the substituting modality, e.g., audition) access to features of the world that are normally accessed through another sensory modality (the substituted modality, e.g., vision). In this article, we address the question of which sensory modality the acquired perception belongs to. We have recourse to the four traditional criteria that have been used to define sensory modalities: sensory organ, stimuli, properties, and qualitative experience (Grice, 1962), (...) to which we have added the criteria of behavioral equivalence (Morgan, 1977), dedication (Keeley, 2002), and sensorimotor equivalence (O’Regan & Noe¨, 2001). We discuss which of them are fulfilled by perception through sensory substitution devices and whether this favors the view that perception belongs to the substituting or to the substituted modality. Though the application of a number of criteria might be taken to point to the conclusion that perception with a sensory substitution device belongs to the substituted modality, we argue that the evidence leads to an alternative view on sensory substitution. According to this view, the experience after sensory substitution is a transformation, extension, or augmentation of our perceptual capacities, rather than being something equivalent or reducible to an already existing sensory modality. We develop this view by comparing sensory substitution devices to other ‘‘mind-enhancing tools’’ such as pen and paper, sketchpads, or calculators. An analysis of sensory substitution in terms of mind-enhancing tools unveils it as a thoroughly transforming perceptual experience and as giving rise to a novel form of perceptual interaction with the environment. (shrink)

We investigate the dynamics of sensory integration for perceiving musical performance, a complex natural behavior. Thirty musically trained participants saw, heard, or both saw and heard, performances by two clarinetists. All participants used a sliding potentiometer to make continuous judgments of tension (a measure correlated with emotional response) and continuous judgments of phrasing (a measure correlated with perceived musical structure) as performances were presented. The data analysis sought to reveal relations between the sensory modalities (vision and audition) (...) and to quantify the effect of seeing the performances on participants' overall subjective experience of the music. In addition to traditional statistics, functional data analysis techniques were employed to analyze time-varying aspects of the data. The auditory and visual channels were found to convey similar experiences of phrasing but different experiences of tension through much of the performances. We found that visual information served both to augment and to reduce the experience of tension at different points in the musical piece (as revealed by functional linear modeling and functional significance testing). In addition, the musicians' movements served to extend the sense of phrasing, to cue the beginning of new phrases, to indicate musical interpretation, and to anticipate changes in emotional content. Evidence for an interaction effect suggests that there may exist an emergent quality when musical performances are both seen and heard. The investigation augments knowledge of human communicative processes spanning language and music, and involving multiple modalities of emotion and information transfer. (shrink)

Many philosophers use findings about sensory substitution devices in the grand debate about how we should individuate the senses. The big question is this: Is “vision” assisted by (tactile) sensory substitution really vision? Or is it tactile perception? Or some sui generis novel form of perception? My claim is that sensory substitution assisted “vision” is neither vision nor tactile perception, because it is not perception at all. It is mental imagery: visual mental imagery triggered by tactile (...) class='Hi'>sensory stimulation. But it is a special form of mental imagery that is triggered by corresponding sensory stimulation in a different sense modality, which I call “multimodal mental imagery.”. (shrink)

It is being increasingly recognized that the Saussurean dictum of “the arbitrariness of the linguistic sign” is in conflict with the pervasiveness of the phenomenon commonly known as “sound symbolism”. After first presenting a historical overview of the debate, however, we conclude that both positions have been exaggerated, and that an adequate explanation of sound symbolism is still lacking. How can there, for example, be (perceived) similarity between expressionsand contents across different sensory modalities? We offer an answer, based (...) on the Peircian notion of iconic ground, and G. Sonesson’s distinction betweenprimary and secondary iconicity. Furthermore, we describe an experimental study, in a paradigm first pioneered by W. Kohler, and recently popularized by V.Ramachandran, in which we varied vowels and consonants in fictive word-forms, and conclude that both types of sounds play a role in perceiving an iconic ground between the word-forms and visual figures. The combination of historical conceptual analysis, semiotic explication and psychological experimentationpresented in this article is characteristic of the emerging paradigm of cognitive semiotics. (shrink)

Synesthesia is a phenomenon in which an additional nonstandard perceptual experience occurs consistently in response to ordinary stimulation applied to the same or another modality. Recent studies suggest an important role of semantic representations in the induction of synesthesia. In the present proposal we try to link the empirically grounded theory of sensory-motor contingency and mirror system based embodied simulation to newly discovered cases of swimming-style color synesthesia. In the latter color experiences are evoked only by showing the synesthetes (...) a picture of a swimming person or asking them to think about a given swimming style. Neural mechanisms of mirror systems seem to be involved here. It has been shown that for mirror-sensory synesthesia, such as mirror-touch or mirror-pain synesthesia, concurrent experiences are caused by the overactivity in the mirror neuron system responding to the specific observation. The comparison of different forms of synesthesia has the potential of challenging conventional thinking on this phenomenon and providing a more general, sensory-motor account of synesthesia encompassing cases driven by semantic or emulational rather than pure sensory or motor representations. (shrink)

Sensory substitution devices make use of one substituting modality to get access to environmental information normally accessed through another modality . Based on behavioural and neuroimaging data, some authors have claimed that using a vision-substituting device results in visual perception. Reviewing these data, we contend that this claim is untenable. We argue that the kind of information processed by a SSD is metamodal, so that it can be accessed through any sensory modality and that the phenomenology associated with (...) the use of a SSD is best described in terms of spatial phenomenology, only. (shrink)

Sensory Individuals: Unimodal and Multimodal Perspectives provides an interdisciplinary, well-balanced, and comprehensive look at different aspects of unisensory and multisensory objects, using both nuanced philosophical analysis and informed empirical work. The research presented in this book represents the field's progression from treating neural sensory processes as primarily modality-specific towards its current state of the art, according to which perception, and its supporting neural processes, are multi-modal, modality-independent, meta-modal, and task-dependent. Even within such approaches sensory stimuli, properties, brain (...) activations, and corresponding perceptual phenomenology can still be characterized in a modality-specific way. The book examines the basic building blocks of human perception, and whether they are best understood as sense modality dependent units of different forms or multimodal perceptual objects. The book combines a variety of innovative and integrative angles to explore the topic and acts as a catalyst for an increasingly diverse field of research, which is in an exciting phase of growth and advancement. New questions are arising as quickly as they are being answered, and the collection Sensory Individuals provides an original and up-to-date addition to the field. (shrink)

This article approaches sensory modalities as semiotically active factors and organizing principles in meaning-making. The focus will be on the special case where modalities mismatch in film – i.e., the soundtrack and visuals present contradictory meanings. The conflict can be characterized by the concept of synthesis that emerges in theories of Eisenstein, Barthes, Jakobson, Lotman, and cognitivists. The artistic functions of such synthesis will be discussed with the help of examples from selected feature films. In the first (...) place, conflicting modalities are inspected in the light of Juri Lotmanʼs theory of two incompatible, but still complementary languages that make up a mechanism for generating new information. In addition, the prospects of evaluating modality conflicts will be touched upon, dismissing synchrony and redundancy as the scale parameters, but acknowledging Lotmanʼs model of space as a primary modeling system that is capable of representing semantic conflicts. (shrink)

Grush's emulation theory comprises both modality-specific and amodal emulators. I suggest that the amodal variety be replaced by multisensory emulators. The key distinction is that multisensory processing retains the characteristics of individual sensory modalities, in contrast to amodal processing. The latter term is better reserved for conceptual and linguistic systems, rather than perception or emulation.