The Visual World Paradigm (VWP) presents listeners with a challenging problem: They must integrate two disparate signals, the spoken language and the visual context, in support of action (e.g., complex movements of the eyes across a scene). We present Impulse Processing, a dynamical systems approach to incremental eye movements in the visual world that suggests a framework for integrating language, vision, and action generally. Our approach assumes that impulses driven by the language and the (...) visual context impinge minutely on a dynamical landscape of attractors corresponding to the potential eye-movement behaviors of the system. We test three unique predictions of our approach in an empirical study in the VWP, and describe an implementation in an artificial neural network. We discuss the Impulse Processing framework in relation to other models of the VWP. (shrink)

Accounts based on the pragmatic maxim of quantity make different predictions about the computation of scalar versus ignorance inferences. These different predictions are evaluated in two eye-tracking experiments using a visual world paradigm to assess the on-line computation of inferences. The test sentences contained disjunction phrases, which engender both kinds of inferences. The first experiment documented that both inferences are computed immediately upon encountering the disjunctive connective, at nearly identical temporal locations. The second experiment was designed to (...) determine whether or not there exists an intermediate stage at which the truth of the corresponding conjunction phrase is ignored. No such stage was found. (shrink)

A number of studies in the extant literature report findings that suggest asymmetry in the way reflexive and pronoun anaphors are interpreted in the early stages of processing: that pronouns are less sensitive to structural constraints, as formulated by Binding Theory, than reflexives, in the initial antecedent retrieval process. However, in previous visual world paradigm eye-tracking studies, these conclusions were based on sentences that placed the critical anaphors within picture noun phrases or prepositional phrases, which have independently (...) been shown not to neatly conform to the Binding Theory principles. We present results from a visual world paradigm eye-tracking experiment that show that when critical anaphors are placed in the indirect object position immediately following a verb as a recipient argument, pronoun and reflexive processing are equally sensitive to structural constraints. (shrink)

We investigated the influence of globally ungrammatical local syntactic constraints on sentence comprehension, as well as the corresponding activation of global and local representations. In Experiment 1, participants viewed visual scenes with objects like a carousel and motorbike while hearing sentences with noun phrase (NP) or verb phrase (VP) modifiers like “The girl who likes the man (from London/very much) will ride the carousel.” In both cases, “girl” and “ride” predicted carousel as the direct object; however, the locally coherent (...) combination “the man from London will ride…” in NP cases alternatively predicted motorbike. During “ride,” local constraints, although ruled out by the global constraints, influenced prediction as strongly as global constraints: While motorbike was fixated less than carousel in VP cases, it was fixated as much as carousel in NP cases. In Experiment 2, these local constraints likewise slowed reading times. We discuss implications for theories of sentence processing. (shrink)

The theoretical notion of ‘construal’ captures the idea that the way in which we describe a scene reflects our conceptualization of it. Relying on the concept of ception – which conjoins conception and perception – we operationalized construal and employed a Visual World Paradigm to establish which aspects of linguistic scene description modulate visual scene perception, thereby affecting event conception. By analysing viewing behaviour after alternating ways of describing location (prepositions), agentivity (active/passive voice) and transfer (NP/pp (...) datives), we found that the linguistic construal of a scene affects its spontaneous visual perception in two ways: either by determining the order in which the components of a scene are accessed or by modulating the distribution of attention over the components, making them more or less salient than they naturally are. We also found evidence for the existence of a cline in the construal effect with stronger expressive differences, such as the prepositional manipulation, inducing more prominent changes in visual perception than the dative manipulation. We discuss the claims language can lay to affecting visual information uptake and hence conceptualization of a static scene in the light of these results. (shrink)

TheConceptual Feature framework predicts that word meaning is represented within a high-dimensional semantic space bounded by weighted contributions of perceptual, affective, and encyclopedic information. The ACF, like latent semantic analysis, is amenable to distance metrics between any two words. We applied predictions of the ACF framework to abstract words using eyetracking via an adaptation of the classical “visual word paradigm”. Healthy adults selected the lexical item most related to a probe word in a 4-item written word array comprising (...) the target and three distractors. The relation between the probe and each of the four words was determined using the semantic distance metrics derived from ACF ratings. Eye movement data indicated that the word that was most semantically related to the probe received more and longer fixations relative to distractors. Importantly, in sets where participants did not provide an overt behavioral response, the fixation rates were nonetheless significantly higher for targets than distractors, closely resembling trials where an expected response was given. Furthermore, ACF ratings which are based on individual words predicted eye fixation metrics of probe-target similarity at least as well as latent semantic analysis ratings which are based on word co-occurrence. The results provide further validation of Euclidean distance metrics derived from ACF ratings as a measure of one facet of the semantic relatedness of abstract words and suggest that they represent a reasonable approximation of the organization of abstract conceptual space. The data are also compatible with the broad notion that multiple sources of information shape the organization of abstract concepts. While the adapted “VWP” is potentially a more metacognitive task than the classical visual world paradigm, we argue that it offers potential utility for studying abstract word comprehension. (shrink)

Two visual world experiments investigated the processing of the implicature associated with some using a “gumball paradigm.” On each trial, participants saw an image of a gumball machine with an upper chamber with orange and blue gumballs and an empty lower chamber. Gumballs dropped to the lower chamber, creating a contrast between a partitioned set of gumballs of one color and an unpartitioned set of the other. Participants then evaluated spoken statements, such as “You got some of (...) the blue gumballs.” Experiment 1 investigated the time course of the pragmatic enrichment from some to not all when the only utterance alternatives available to refer to the different sets were some and all. In Experiment 2, the number terms two, three, four, and five were also included in the set of alternatives. Scalar implicatures were delayed relative to the interpretation of literal statements with all only when number terms were available. The results are interpreted as evidence for a constraint-based account of scalar implicature processing. (shrink)

The Abstract Conceptual Feature (ACF) framework predicts that word meaning is represented within a high‐dimensional semantic space bounded by weighted contributions of perceptual, affective, and encyclopedic information. The ACF, like latent semantic analysis, is amenable to distance metrics between any two words. We applied predictions of the ACF framework to abstract words using eyetracking via an adaptation of the classical “visual word paradigm” (VWP). Healthy adults (n = 20) selected the lexical item most related to a probe word (...) in a 4‐item written word array comprising the target and three distractors. The relation between the probe and each of the four words was determined using the semantic distance metrics derived from ACF ratings. Eye movement data indicated that the word that was most semantically related to the probe received more and longer fixations relative to distractors. Importantly, in sets where participants did not provide an overt behavioral response, the fixation rates were nonetheless significantly higher for targets than distractors, closely resembling trials where an expected response was given. Furthermore, ACF ratings which are based on individual words predicted eye fixation metrics of probe‐target similarity at least as well as latent semantic analysis ratings which are based on word co‐occurrence. The results provide further validation of Euclidean distance metrics derived from ACF ratings as a measure of one facet of the semantic relatedness of abstract words and suggest that they represent a reasonable approximation of the organization of abstract conceptual space. The data are also compatible with the broad notion that multiple sources of information (not restricted to sensorimotor and emotion information) shape the organization of abstract concepts. While the adapted “VWP” is potentially a more metacognitive task than the classical visual world paradigm, we argue that it offers potential utility for studying abstract word comprehension. (shrink)

Previous research has shown that language comprehenders resolve reference quickly and incrementally, but not much is known about the neural processes and representations that are involved. Studies of visual short-term memory suggest that access to the representation of an item from a previously seen display is associated with a negative evoked potential at posterior electrodes contralateral to the spatial location of that item in the display. In this paper we demonstrate that resolving the reference of a noun phrase in (...) a recently seen visual display is associated with an event-related potential that is analogous to this effect. Our design was adapted from the visual world paradigm: in each trial, participants saw a display containing three simple objects, followed by a question about the objects, such as Was the pink fish next to a boat?, presented word by word. Questions differed in whether the color adjective allowed the reader to identify the referent of the noun phrase or not (i.e., whether one or more objects of the named color were present). Consistent with our hypothesis, we observed that reference resolution by the adjective was associated with a negative evoked potential at posterior electrodes contralateral to spatial location of the referent, starting approximately 333 ms after the onset of the adjective. The fact that the laterality of the effect depended upon the location of the referent within the display suggests that reference resolution in visual domains involves, at some level, a modality-specific representation. In addition, the effect gives us an estimate of the time course of processing from perception of the written word to the point at which its meaning is brought into correspondence with the referential domain. (shrink)

We investigated the effects of everyday language exposure on the prediction of orthographic and phonological forms of a highly predictable word during listening comprehension. Native Japanese speakers in Tokyo (Experiment 1) and Berlin (Experiment 2) listened to sentences that contained a predictable word and viewed four objects. The critical object represented the target word (e.g., /sakana/;fish), an orthographic competitor (e.g., /tuno/;horn), a phonological competitor (e.g., /sakura/;cherry blossom), or an unrelated word (e.g., /hon/;book). The three other objects were distractors. The Tokyo (...) group fixated the target and the orthographic competitor over the unrelated objects before the target word was mentioned, suggesting that they pre-activated the orthographic form of the target word. The Berlin group showed a weaker bias toward the target than the Tokyo group, and they showed a tendency to fixate the orthographic competitor only when the orthographic similarity was very high. Thus, prediction effects were weaker in the Berlin group than in the Tokyo group. We found no evidence for the prediction of phonological information. The obtained group differences support probabilistic models of prediction, which regard the built-up language experience as a basis of prediction. (shrink)

The human sentence processor is able to make rapid predictions about upcoming linguistic input. For example, upon hearing the verb eat, anticipatory eye-movements are launched toward edible objects in a visual scene. However, the cognitive mechanisms that underlie anticipation remain to be elucidated in ecologically valid contexts. Previous research has, in fact, mainly used clip-art scenes and object arrays, raising the possibility that anticipatory eye-movements are limited to displays containing a small number of objects in a visually impoverished context. (...) In Experiment 1, we confirm that anticipation effects occur in real-world scenes and investigate the mechanisms that underlie such anticipation. In particular, we demonstrate that real-world scenes provide contextual information that anticipation can draw on: When the target object is not present in the scene, participants infer and fixate regions that are contextually appropriate. Experiment 2 investigates whether such contextual inference requires the co-presence of the scene, or whether memory representations can be utilized instead. The same real-world scenes as in Experiment 1 are presented to participants, but the scene disappears before the sentence is heard. We find that anticipation occurs even when the screen is blank, including when contextual inference is required. We conclude that anticipatory language processing is able to draw upon global scene representations to make contextual inferences. These findings are compatible with theories assuming contextual guidance, but posit a challenge for theories assuming object-based visual indices. (shrink)

Modulation of visual attention in the Visual World Paradigm relies on parallel processing of linguistic and visual information. Previous studies have argued that the human linguistic capacity includes an aspect of anticipation of upcoming material. Such anticipation can be triggered by both lexical and grammatical/morphosyntactic cues. In this study, we investigated the relationship between comprehension and prediction by testing how subtle changes in visual representations can affect the processing of grammatical case cues in Russian (...) by Russian-German bilingual children (n = 49, age 8−13). The linguistic manipulation followed previous designs, contrasting SVO and OVS sentences, where the first NP (NP1) was marked with nominative or accusative case, respectively. Three types of visual displays were compared: (i) individual referents (potential agent/theme); (ii) pairs of referents (NP1 + potential agent/theme); and (iii) events (representing interactions between the referents). Participants were significantly more sensitive to the case manipulation when presented with events compared to the other two types of visual display. This suggests that they were able to quickly integrate the thematic role information signaled by grammatical case in the event representations. However, they were less likely to use the case information to anticipate upcoming arguments when the target pictures represented individual referents or pairs of noninteracting referents. We hypothesize that the process of argument anticipation is mediated by the activation of syntactic templates (SVO or OSV, depending on the case marking on NP1). The relatively weak anticipation effect observed may be attributed to the absence, or weak representation, of the noncanonical OVS template in the bilingual children's long-term memory. (shrink)

Recent studies have documented substantial variability among typical listeners in how gradiently they categorize speech sounds, and this variability in categorization gradience may link to how listeners weight different cues in the incoming signal. The present study tested the relationship between categorization gradience and cue weighting across two sets of English contrasts, each varying orthogonally in two acoustic dimensions. Participants performed a four‐alternative forced‐choice identification task in a visual world paradigm while their eye movements were monitored. We (...) found that (a) greater categorization gradience derived from behavioral identification responses corresponds to larger secondary cue weights derived from eye movements; (b) the relationship between categorization gradience and secondary cue weighting is observed across cues and contrasts, suggesting that categorization gradience may be a consistent within‐individual property in speech perception; and (c) listeners who showed greater categorization gradience tend to adopt a buffered processing strategy, especially when cues arrive asynchronously in time. (shrink)

In infancy, maternal socioeconomic status is associated with real-time language processing skills, but whether or not this relationship carries into adulthood is unknown. We explored the effects of maternal SES in college-aged adults on eye-tracked, spoken sentence comprehension tasks using the visual world paradigm. When sentences ended in highly plausible, expected target nouns, higher SES was associated with a greater likelihood of considering alternative endings related to the action of the sentence. Moreover, for unexpected sentence endings, individuals (...) from higher SES backgrounds were sensitive to whether the ending was action-related or unrelated, showing a benefit for plausible endings. Individuals from lower SES backgrounds did not show this advantage. This suggests maternal SES can influence the dynamics of sentence processing even in adulthood, with consequences for processing unexpected content. These findings highlight the importance of early lexical experience for adult language skills. (shrink)

Predicates like “coloring‐the‐star” denote events that have a temporal duration and a culmination point (telos). When combined with perfective aspect (e.g., “Valeria has colored the star”), a culmination inference arises implying that the action has stopped, and the star is fully colored. While the perfective aspect is known to constrain the conceptualization of the event as telic, many reading studies have demonstrated that readers do not make early commitments as to whether the event is bounded or unbounded. A few (...) class='Hi'>visual‐world studies tested the processing of telic predicates during online sentence processing, demonstrating an early integration of aspectual and temporal cues. By employing the visual‐world paradigm, we tested the incremental processing of the perfective aspect in Italian in two eye‐tracking studies in which listeners heard durative predicates in the perfective form in a scenario showing a completed and a non‐completed event. Differently from previous studies, we compared telic durative predicates such as “coloring‐the‐star” to punctual predicates such as “lighting‐the‐candle.” While for punctual predicates, the inferences of telicity (the event has a telos) and of culmination (the telos is reached) are lexically encoded in the perfective verb, for durative predicates, the degree of event completion (visually encoded) needs to be integrated with perfective aspect (linguistically encoded) for the culmination inference derivation. By modulating the interaction of visual and linguistic stimuli across the two experiments, we show that the verb's perfective aspect triggers the culmination inference incrementally during sentence processing, offering novel evidence for the continuous integration of linguistic processing with real‐world visual information. (shrink)

Across languages, children map words to meaning with great efficiency, despite a seemingly unconstrained space of potential mappings. The literature on how children do this is primarily limited to spoken language. This leaves a gap in our understanding of sign language acquisition, because several of the hypothesized mechanisms that children use are visual (e.g., visual attention to the referent), and sign languages are perceived in the visual modality. Here, we used the Human Simulation Paradigm in American (...) Sign Language (ASL) to determine potential cues to word learning. Sign‐naïve adult participants viewed video clips of parent–child interactions in ASL, and at a designated point, had to guess what ASL sign the parent produced. Across two studies, we demonstrate that referential clarity in ASL interactions is characterized by access to information about word class and referent presence (for verbs), similarly to spoken language. Unlike spoken language, iconicity is a cue to word meaning in ASL, although this is not always a fruitful cue. We also present evidence that verbs are highlighted well in the input, relative to spoken English. The results shed light on both similarities and differences in the information that learners may have access to in acquiring signed versus spoken languages. (shrink)

Age has been shown to influence language comprehension, with delays, for instance, in older adults' expectations about upcoming information. We examined to what extent expectations about upcoming event information change across the lifespan and as a function of different world-language relations. In a visual-world paradigm, participants in all three age groups inspected a speaker whose facial expression was either smiling or sad. Next they inspected two clipart agents depicted as acting upon a patient. Control scenes featured (...) the same three characters without the action depictions. While inspecting the depictions, comprehenders listened to a German sentence [e.g., Den Marienkäfer kitzelt vergnügt der Kater; literally: “The ladybug tickles happily the cat ”]. Referential verb-action relations could, in principle, cue the cat-agent and so could non-referential relations via links from the speaker's smile to “happily” and the cat's smile. We examined variation in participants' visual anticipation of the agent before it was mentioned depending on participant age and whether the referentially mediated action depiction or the non-referentially associated speaker smile cued the agent. The action depictions rapidly boosted participants' visual anticipation of the agent, facilitating thematic role assignment in all age groups. By contrast, effects of the non-referentially cued speaker smile emerged in the younger adults only. We outline implications of these findings for processing accounts of the temporally coordinated interplay between listeners' age-dependent language comprehension, their interrogation of the visual context, and visual context influences. (shrink)

Evidence from numerous studies using the visual world paradigm has revealed both that spoken language can rapidly guide attention in a related visual scene and that scene information can immediately influence comprehension processes. These findings motivated the coordinated interplay account (Knoeferle & Crocker, 2006) of situated comprehension, which claims that utterance‐mediated attention crucially underlies this closely coordinated interaction of language and scene processing. We present a recurrent sigma‐pi neural network that models the rapid use of scene (...) information, exploiting an utterance‐mediated attentional mechanism that directly instantiates the CIA. The model is shown to achieve high levels of performance (both with and without scene contexts), while also exhibiting hallmark behaviors of situated comprehension, such as incremental processing, anticipation of appropriate role fillers, as well as the immediate use, and priority, of depicted event information through the coordinated use of utterance‐mediated attention to the scene. (shrink)

The aim of this study was to find out how people process the dialectal variation encountered in the daily linguistic input. We conducted an eye tracking study (Visual Word Paradigm) that targeted the predictive processing of grammatical gender markers. Three different groups of Norwegian speakers took part in the experiment: one group of students from the capital Oslo, and two groups of dialect speakers from the Western Norwegian town Sogndal. One Sogndal group was defined as ``stable dialect speakers'', (...) and one as ``unstable dialect speakers'', based on a background questionnaire. The students participated in two eye tracking experiments each, one conducted in the their own dialect, and one in the other dialect (i.e., Sogn dialect for the Oslo students, and Oslo dialect for the Sogndal students). The gender systems in the two dialects differ: the Sogn dialect makes an obligatory three-gender split (Masculine, Feminine and Neuter) whereas the Oslo dialect only obligatorily makes a two gender distinction. The research question was whether speakers could use gender markers to predict the upcoming target noun in both local and non-local dialect mode, and furthermore, if they correctly could adjust their expectations based on dialect mode. The results showed that the Sogn speakers could predict upcoming linguistic material both in the local and Oslo dialect, but only the stable group were able to adjust their predictions based on the dialect mode. The unstable group applied a more general Oslo-compatible parsing to both the local and the non-local dialect. The Oslo speakers on the other hand were able to use gender markers as predictors only in their own dialect. We argue that the stable Sogn group should be treated as a bilingual group, as they show native-like skills in both varieties, while the unstable Sogn group can be seen as accommodated monolinguals, in that they treat the two varieties as sharing an underspecified grammar. The Oslo group on the other hand lacks sufficient competence in the other dialect to make use of grammatical markers to make predictions. (shrink)

Referential gaze has been shown to benefit language processing in situated communication in terms of shifting visual attention and leading to shorter reaction times on subsequent tasks. The present study simultaneously assessed both visual attention and, importantly, the immediate cognitive load induced at different stages of sentence processing. We aimed to examine the dynamics of combining visual and linguistic information in creating anticipation for a specific object and the effect this has on language processing. We report evidence (...) from three visual‐world eye‐tracking experiments, showing that referential gaze leads to a shift in visual attention toward the cued object, which consequently lowers the effort required for processing the linguistic reference. Importantly, perceiving and following the gaze cue did not prove costly in terms of cognitive effort, unless the cued object did not fit the verb selectional preferences. (shrink)

This study investigates whether listeners’ cue weighting predicts their real‐time use of asynchronous acoustic information in spoken word recognition at both group and individual levels. By focusing on the time course of cue integration, we seek to distinguish between two theoretical views: the associated view (cue weighting is linked to cue integration strategy) and the independent view (no such relationship). The current study examines Seoul Korean listeners’ (n = 62) weighting of voice onset time (VOT, available earlier in time) and (...) onset fundamental frequency of the following vowel (F0, available later in time) when perceiving Korean stop contrasts (Experiment 1: cue‐weighting perception task) and the timing of VOT integration when recognizing Korean words that begin with a stop (Experiment 2: visual‐world eye‐tracking task). The group‐level results reveal that the timing of the early cue (VOT) integration is delayed when the later cue (F0) serves as the primary cue to process the stop contrast, supporting a relationship between cue weighting and the timing of cue integration (the associated view). At the individual level, listeners with greater reliance on F0 than VOT exhibited a further delayed integration of VOT. These findings suggest that the real‐time processing of asynchronously occurring acoustic cues for lexical activation is modulated by the weight that listeners assign to those cues, providing evidence for the associated view of cue integration. This study offers insights into the mechanisms of cue integration and spoken word recognition, and they shed light on variability in cue integration strategies among listeners. (shrink)

The aim of this paper is to show how a sensory approach to visual arts can be relevant for philosophy and how this prism, once brought to philosophy, can give insights on art in return. I will try to demonstrate that the difference between modernity’s two main schools of thought (namely, materialism and idealism) can be understood – thanks to the model of painting as an allegory of the world – as an exclusive preference for one sense: touch (...) for materialism and sight for idealism. Much as the two schools differ, both consider the painting (or the world) a finite picture, whose elements can be juxtaposed in a single homogeneous plane of knowledge devoid of any opacity. This leads to separate both vision and the mind from the body and from the world. As a result, sight and touch end up dissociated. That is why, to challenge this modern paradigm and renew with a holistic relationship to the world and to ourselves, philosophers and artists propose a shift towards a synaesthetic approach to painting. Such an approach will uncover a modality of being where the body, the subject, and the world, as well as sight and touch, can be reunited. Thus, I will show that Merleau-Ponty’s ontology of the flesh – that echoes through the paintings of artists who attempt to break free from the frame and engage the observer with all his senses and being, such as Racławice Panorama, Malevich, El Lissitzky, Cézanne or the Navajos’ sand paintings – can offer, thanks to a synaesthetic approach, both a new framework for philosophy and a new understanding of visual arts. (shrink)

This study investigated the perception of Mandarin tonal alternations in disyllabic words. In Mandarin, a low-dipping Tone3 is converted to a high-rising Tone2 when followed by another Tone3, known as third tone sandhi. Although previous studies showed statistically significant differences in F0 between a high-rising Sandhi-Tone3 and a Tone2, native Mandarin listeners failed to correctly categorize these two tones in perception tasks. The current study utilized the visual-world paradigm in eye-tracking to further examine whether acoustic details in (...) lexical tone aid lexical access in Mandarin. Results showed that Mandarin listeners tend to process Tone2 as Tone2 whereas they tend to first process Sandhi-T3 as both Tone3 and Tone2, then later detect the acoustic differences between the two tones revealed by the sandhi context, and finally activate the target word during lexical access. The eye-tracking results suggest that subtle acoustic details of F0 may facilitate lexical access in automatic fashion in a tone language. (shrink)

In lieu of an abstract, here is a brief excerpt of the content:Rembrandt’s Art: A Paradigm for Critical Thinking and AestheticsMark S. Conn (bio)IntroductionThe purpose of art is to lay bare the questions, which have been hidden by the answers.—James BaldwinPhilosophers have asked, How do we know the world? Over centuries, many visual artists have responded to this question by provoking us to see the world differently—through their own eyes. Rembrandt, by no small measure, is one (...) of those artists. While many people have attempted to understand and interpret the art of this quintessential master, few have used his art as a tool to teach critical thinking. The challenge, then, is to set forth a paradigm, using Rembrandt’s art as the vehicle, to teach critical thinking skills. These skills can be applied across the curriculum without losing an appreciation for the aesthetic characteristics of his art.Several sections in this article begin with a foundational discussion of the connection between art and the general curriculum, including how the effectiveness of that curriculum may then be measured. Continuing with a working definition of critical thinking, I demonstrate how Rembrandt’s work relates particularly well to the social studies curriculum. Definitions of critical thinking skills are discussed just prior to introducing the paradigm. A discussion of the paradigm’s relationship to aesthetics and necessary concepts is followed by the conclusion, which contains the graphic illustration of the paradigm. [End Page 68]Foundational DiscussionIn order to create a paradigm that encompasses Rembrandt’s art, critical thinking, and aesthetics, it is important to understand the connection that art has to the curriculum. By examining the way in which this relationship has evolved and is measured, we discover its foundation. It also produces a rationale for the paradigm.For decades the educational community has debated the validity of including arts in the curriculum. There are those who believe that in order to justify inclusion in the curriculum, there needs to be evidence that art has a quantifiable impact upon learning. Others believe that there is a need to teach art for art’s sake.1 Elliot Eisner stated, “More often than we would like, arts educators receive requests to justify their professional existence or existence of the arts in our schools on the basis of their contribution to non-art outcomes.”2 According to Eisner’s research, there have been no studies that show a statistically significant relationship between the arts and academic success.3 Additionally, in their final report evaluators from Transforming Education Through the Arts Challenge (TETAC) concluded that “Analysis of site-specific data found no significant relationship between program implementation and performance on reading and math tests.”4 There are many sources that can be cited that report the same findings. In assessing these kinds of findings we find the conclusions to be approximately the same.If we look at the U.S. Department of Education’s National Assessment of Educational Progress, which each year releases “The Nation’s Report Card,” we find a tremendous amount of data.5 However, most of what we find is data that is analyzed scientifically, with a long but incomplete list of incomplete variables. Therefore, it is difficult to come to any consensus about which programs, in which states, are successful and why. All we actually see is who is supposedly doing better in various curriculum areas based upon standardized tests that vary from state to state. Consider an example of current thinking regarding curriculum and testing from an article by Diane Ravitch, former assistant U.S. secretary of education, and Deborah Meir, senior scholar at New York University’s Steinhardt School of Education:Both of us acknowledged that our choices involve risks. A national curriculum might be unwieldy and superficial (“a mile wide and an inch deep—ironically the charge directed at our current incoherent and fragmented curriculum”) as well as politically compromised, while a local one might reflect the low expectations of the local community as local foolishness and local biases (some schools, for example, might teach intelligent design). We agreed that the measurement of “results”—what constitutes intellectual achievement—has been badly distorted by current local and state tests, which undermine high-quality tasks and make... (shrink)

A classic problem in cognitive science concerns how listeners perceive and understand speech as comprised of discrete words. We propose a Syllable Inference account of spoken word recognition and segmentation, under which alternative hierarchical models of syllables, words, and phonemes are dynamically posited from cues that include current and past speech rate, with a goal of maximal prediction of sensory input. Three experiments using the Visual World eye‐tracking paradigm provide evidence supporting our proposal.

This article presents a fresh perspective on the interplay among visual, verbal, and auditory modalities, positing that these modalities, as semogenic resources, compete to express dynamic meanings. The theoretical paradigm emphasizes that whether a modality or an element within a modality gets or loses semantic status, it will elicit an additional layer of social meaning to depict a comprehensive picture of a story together with an explicit semiotic meaning. The article adopts a qualitative method to analyze the data, (...) which are drawn from The Good Wife and My Roommate is a Gumiho and annotated in ELAN 6.3. It was found that modal competition can shed light on the dynamic meaning-making processes in semiotic and societal orientations. Modal competition may distort space and time of different stories, and reconstruct a different discursive spatio-temporal dimension in the TV world. It can diversify the dynamic orientations from New to Given in visual, verbal, and auditory texts of multimodal discourses to tell stories. Modal competition provides a lens to understand the multidimensional reality and to appreciate the aesthetics of a modern TV series. (shrink)

Speech unfolds over time, and the cues for even a single phoneme are rarely available simultaneously. Consequently, to recognize a single phoneme, listeners must integrate material over several hundred milliseconds. Prior work contrasts two accounts: (a) a memory buffer account in which listeners accumulate auditory information in memory and only access higher level representations (i.e., lexical representations) when sufficient information has arrived; and (b) an immediate integration scheme in which lexical representations can be partially activated on the basis of early (...) cues and then updated when more information arises. These studies have uniformly shown evidence for immediate integration for a variety of phonetic distinctions. We attempted to extend this to fricatives, a class of speech sounds which requires not only temporal integration of asynchronous cues (the frication, followed by the formant transitions 150–350 ms later), but also integration across different frequency bands and compensation for contextual factors like coarticulation. Eye movements in the visual world paradigm showed clear evidence for a memory buffer. Results were replicated in five experiments, ruling out methodological factors and tying the release of the buffer to the onset of the vowel. These findings support a general auditory account for speech by suggesting that the acoustic nature of particular speech sounds may have large effects on how they are processed. It also has major implications for theories of auditory and speech perception by raising the possibility of an encapsulated memory buffer in early auditory processing. (shrink)

This study investigated the relative role of sub-syllabic components in spoken word recognition of Mandarin Chinese using an eye-tracking experiment with a visual world paradigm. Native Mandarin speakers were presented with four pictures and an auditory stimulus. They were required to click the picture according to the sound stimulus they heard, and their eye movements were tracked during this process. For a target word, nine conditions of competitors were constructed in terms of the amount of their phonological (...) overlap with the target: consonant competitor, rime competitor, tone competitor, consonant plus rime competitor, consonant plus tone competitor, rime plus tone competitor, cohort competitor, cohort plus tone competitor, and baseline competitor. A growth curve analysis was conducted with the fixation to competitors, targets, and distractors, and the results showed that competitors with consonant or rime overlap can be adequately activated, while tone overlap plays a weaker role since additional tonal information can strengthen the competitive effect only when it was added to a candidate that already bears much phonological similarity with the target. Mandarin words are processed in an incremental way in the time course of word recognition since different partially overlapping competitors could be activated immediately; like the pattern found in English, both cohort and rime competitors were activated to compete for lexical activation, but these two competitors were not temporally distinctive and mainly differed in the size of their competitive effects. Generally, the gradation of activation based on the phonological similarity between target and candidates found in this study was in line with the continuous mapping models and may reflect a strategy of native speakers shaped by the informative characteristics of the interaction among different sub-syllabic components. (shrink)

This study investigated grammatical gender processing in school-age Spanish-English bilingual children using a visual world paradigm with a 4-picture display where the target noun was heard with a gendered article that was either in a context where all distractor images were the same gender as the target noun or in a context where all distractor images were the opposite gender than the target noun. We investigated 32 bilingual children who were exposed to Spanish since infancy and began (...) learning English by school entry. Along with the eye-tracking experiment, all children participated in a standardized language assessment and told narratives in English and Spanish, and parents reported on their child's current Spanish language use. The differential proportion fixations to target were analyzed in two time regions with linear mixed-effects models. Results show that prior to the target word being spoken, these bilingual children did not use the gendered articles to actively anticipate upcoming nouns. In the subsequent time region, it was shown that there are differences in the way they use feminine and masculine articles, with a lack of use of the masculine article and a potential facilitatory use of the feminine article for children who currently use more Spanish than English. This asymmetry in the use of gendered articles in processing is modulated by current Spanish language use and trends with results found for bilingual and second-language learning adults. (shrink)

This paper presents the results of an eyetracking study that uses the Visual World Paradigm to determine whether heritage speakers of Polish can use grammatical gender cues to facilitate lexical retrieval of the subsequent noun during real time processing. Previous work has investigated this question for heritage speakers of Spanish with gender cues located on definite articles, which are highly frequent in Spanish; the results are therefore consistent both with a grammatical account, wherein heritage speakers access abstract (...) syntactic gender features during processing, and a probabilistic account, wherein facilitation is due to transition probabilities between frequently co-occurring elements. In Polish, gender cues appear on adjectives, which are optional and infrequent. Results of the present study show that heritage speakers of Polish can use gender on inflected adjectives to fixate on the target noun faster in trials where that gender cue uniquely identifies the target noun. This finding supports a grammatical rather than probabilistic account of the facilitative use of grammatical gender in this population: heritage speakers are able to access abstract syntactic information in real time to aid word recognition in a target-like manner. (shrink)

Based on electrophysiological and anatomical studies, a prevalent conception is that the visual system recovers features of the world from retinal images to generate perceptions and guide behavior. This paradigm, however, is unable to explain why visual perceptions differ from physical measurements, or how behavior could routinely succeed on this basis. An alternative is that vision does not recover features of the world, but assigns perceptual qualities empirically by associating frequently occurring stimulus patterns with useful (...) responses on the basis of survival and reproductive success. The purpose of the present article is to briefly describe this strategy of vision and the evidence for it. (shrink)

Difficulties understanding speech form one of the most prevalent complaints among older adults. Successful speech perception depends on top-down linguistic and cognitive processes that interact with the bottom-up sensory processing of the incoming acoustic information. The relative roles of these processes in age-related difficulties in speech perception, especially when listening conditions are not ideal, are still unclear. In the current study, we asked whether older adults with a larger working memory capacity process speech more efficiently than peers with lower capacity (...) when speech is presented in noise, with another task performed in tandem. Using the Eye-tracking of Word Identification in Noise Under Memory Increased Load an adapted version of the “visual world” paradigm, 36 older listeners were asked to follow spoken instructions presented in background noise, while retaining digits for later recall under low or high memory load. In critical trials, instructions directed listeners’ gaze to pictures of objects whose names shared onset or offset sounds with the name of a competitor that was displayed on the screen at the same time. We compared listeners with different memory capacities on the time course for spoken word recognition under the two memory loads by testing eye-fixations on a named object, relative to fixations on an object whose name shared phonology with the named object. Results indicated two trends. For older adults with lower working memory capacity, increased memory load did not affect online speech processing, however, it impaired offline word recognition accuracy. The reverse pattern was observed for older adults with higher working memory capacity: increased task difficulty significantly decreases online speech processing efficiency but had no effect on offline word recognition accuracy. Results suggest that in older adults, adaptation to adverse listening conditions is at least partially supported by cognitive reserve. Therefore, additional cognitive capacity may lead to greater resilience of older listeners to adverse listening conditions. The differential effects documented by eye movements and accuracy highlight the importance of using both online and offline measures of speech processing to explore age-related changes in speech perception. (shrink)

We present two eye-tracking experiments on the interpretation of sentences like “The tall girl is the only one that …,” which are ambiguous between the anaphoric and the exophoric interpretation. These interpretations differ in informativeness: in a positive context, the exophoric reading entails the anaphoric, while in a negative context the entailment pattern is reversed and the anaphoric reading is the strongest one. We tested whether adults rely on considerations about informativeness in solving the ambiguity. The results show that participants (...) interpreted one exophorically in both positive and negative contexts. Given these findings, we cast doubts on the idea that Informativeness plays a role in ambiguity resolution and proposes a Principle of Maximal Exploitation: When a context is provided, adults extend their domain of evaluation to include the whole scenario, independently from truth-conditional considerations about informativity and strength. (shrink)

Recent laboratory experiments have shown that both infant and adult learners can acquire word‐referent mappings using cross‐situational statistics. The vast majority of the work on this topic has used unfamiliar objects presented on neutral backgrounds as the visual contexts for word learning. However, these laboratory contexts are much different than the real‐world contexts in which learning occurs. Thus, the feasibility of generalizing cross‐situational learning beyond the laboratory is in question. Adapting the Human Simulation Paradigm, we conducted a (...) series of experiments examining cross‐situational learning from children's egocentric videos captured during naturalistic play. Focusing on individually ambiguous naming moments that naturally occur during toy play, we asked how statistical learning unfolds in real time through accumulating cross‐situational statistics in naturalistic contexts. We found that even when learning situations were individually ambiguous, learners’ performance gradually improved over time. This improvement was driven in part by learners’ use of partial knowledge acquired from previous learning situations, even when they had not yet discovered correct word‐object mappings. These results suggest that word learning is a continuous process by means of real‐time information integration. (shrink)

In this article I discuss the prosthetic phenomenon during the First World War and Weimar Germany. As opposed to contemporary trends, with their inflationary use of the ‘prosthesis’, sometimes even hypothesizing ‘prostheticization’ as a paradigm, I seek to return the debate about the prosthesis to its historical concreteness. I describe the phenomenology of the prosthesis in three senses: first, in the statistical sense, in the form of a dramatic growth in the number of prostheses; second, in the (...) class='Hi'>visual sense, in the form of a dramatic growth in the visibility of the prosthesis. Basing myself on the Heideggerian perception of the ‘phenomenon’, I seek to reveal an additional, third, aspect of the phenomenology of the prosthesis. It is my contention, against the background of the major catastrophe of the First World War and the frequent crises that afflicted Weimar Germany, but also in the light of additional contexts — technological, economic, cultural — that the prosthesis was increasingly perceived as a phenomenon, i.e. as something which appeared in a wide range of ways — as prosthesis, as tool (hammer, writing instrument), as an organic limb (hand, leg), and even as a paradigm (man as ‘prosthetic God’, man as ‘Dasein’). (shrink)

Computational modeling and eye-tracking were used to investigate how phonological and semantic information interact to influence the time course of spoken word recognition. We extended our recent models to account for new evidence that competition among phonological neighbors influences activation of semantically related concepts during spoken word recognition . The model made a novel prediction: Semantic input modulates the effect of phonological neighbors on target word processing, producing an approximately inverted-U-shaped pattern with a high phonological density advantage at an intermediate (...) level of semantic input—in contrast to the typical disadvantage for high phonological density words in spoken word recognition. This prediction was confirmed with a new analysis of the Apfelbaum et al. data and in a visual world paradigm experiment with preview duration serving as a manipulation of strength of semantic input. These results are consistent with our previous claim that strongly active neighbors produce net inhibitory effects and weakly active neighbors produce net facilitative effects. (shrink)

Embodied theories of language postulate that language meaning is stored in modality-specific brain areas generally involved in perception and action in the real world. However, the temporal dynamics of the interaction between modality-specific information and lexical-semantic processing remain unclear. We investigated the relative timing at which two types of modality-specific information (action-based and visual-form information) contribute to lexical-semantic comprehension. To this end, we applied a behavioral priming paradigm in which prime and target words were related with respect (...) to (1) action features, (2) visual features, or (3) semantically associative information. Using a Go/No-Go lexical decision task, priming effects were measured across four different inter-stimulus intervals (ISI = 100, 250, 400, and 1000 ms) to determine the relative time course of the different features. Notably, action priming effects were found in ISIs of 100, 250, and 1000 ms whereas a visual priming effect was seen only in the ISI of 1000 ms. Importantly, our data suggest that features follow different time courses of activation during word recognition. In this regard, feature activation is dynamic, measurable in specific time windows but not in others. Thus the current study (1) demonstrates how multiple ISIs can be used within an experiment to help chart the time course of feature activation and (2) provides new evidence for embodied theories of language. (shrink)

The new non-fiction picturebook for children is conceived not just as an informational book, but first and foremost as a beautiful object, characterized by a largely visual and proudly creative approach to knowledge. By blending information and artistic illustration/design, transmission of data and sophisticated aesthetic experimentation, this medium seems to bring successfully together the rational/explicit and the aesthetic/intuitive way of attending to the world, with promising consequences for the development of an integrated learning experience. Applying the findings of (...) cognitive sciences to the analysis of non-fiction picturebooks can be enlightening to understand the full potential of such books in sharing knowledge with children. Their intrinsic multimodality and ability to stimulate different parts of the reader’s brain makes them valuable in any educational context revolving around the paradigm of complexity and having connection (as opposed to disjunction) of different levels of experience as its aim. (shrink)