Using an asymptotic characterization of probabilistic finite state languages over a one-letter alphabet we construct a probabilistic language with regular support that cannot be generated by probabilistic CFGs. Since all probability values used in the example are rational, our work is immune to the criticism leveled by Suppes (Synthese 22:95–116, 1970 ) against the work of Ellis ( 1969 ) who first constructed probabilistic FSLs that admit no probabilistic FSGs. Some implications for probabilistic language (...) modeling by HMMs are discussed. (shrink)

We introduce a set of biologically and computationally motivated design choices for modeling the learning of language, or of other types of sequential, hierarchically structured experience and behavior, and describe an implemented system that conforms to these choices and is capable of unsupervised learning from raw natural-language corpora. Given a stream of linguistic input, our model incrementally learns a grammar that captures its statistical patterns, which can then be used to parse or generate new data. The grammar constructed in this (...) manner takes the form of a directed weighted graph, whose nodes are recursively (hierarchically) defined patterns over the elements of the input stream. We evaluated the model in seventeen experiments, grouped into five studies, which examined, respectively, (a) the generative ability of grammar learned from a corpus of natural language, (b) the characteristics of the learned representation, (c) sequence segmentation and chunking, (d) artificial grammar learning, and (e) certain types of structure dependence. The model's performance largely vindicates our design choices, suggesting that progress in modeling language acquisition can be made on a broad front—ranging from issues of generativity to the replication of human experimental findings—by bringing biological and computational considerations, as well as lessons from prior efforts, to bear on the modeling approach. (shrink)

While rules and exemplars are usually viewed as opposites, this paper argues that they form end points of the same distribution. By representing both rules and exemplars as (partial) trees, we can take into account the fluid middle ground between the two extremes. This insight is the starting point for a new theory of language learning that is based on the following idea: If a language learner does not know which phrase‐structure trees should be assigned to initial sentences, s/he allows (...) (implicitly) for all possible trees and lets linguistic experience decide which is the “best” tree for each sentence. The best tree is obtained by maximizing “structural analogy” between a sentence and previous sentences, which is formalized by the most probable shortest combination of subtrees from all trees of previous sentences. Corpus‐based experiments with this model on the Penn Treebank and the Childes database indicate that it can learn both exemplar‐based and rule‐based aspects of language, ranging from phrasal verbs to auxiliary fronting. By having learned the syntactic structures of sentences, we have also learned the grammar implicit in these structures, which can in turn be used to produce new sentences. We show that our model mimicks children’s language development from item‐based constructions to abstract constructions, and that the model can simulate some of the errors made by children in producing complex questions. (shrink)

This article discusses the results of an investigation on discourse representation in a corpus of 34 million words constituted by texts in Brazilian Portuguese from two different genres: the research article and the science popularization article. Drawing on a systemic functional grammar perspective of language and pursuing a probabilistic approach, it focuses on the realization of lexicogrammatical systems of direct discourse representation as enacting interpersonal and social relationships. It is argued that the citation practices employed by writers in the (...) genres studied negotiate discursive distance and are influenced by the status culturally and historically ascribed to scientific knowledge. (shrink)

We examine the expressive power of probabilistic context free grammars (PCFGs), with a special focus on the use of probabilities as a mechanism for reducing ambiguity by filtering out unwanted parses. Probabilities in PCFGs induce an ordering relation among the set of trees that yield a given input sentence. PCFG parsers return the trees bearing the maximum probability for a given sentence, discarding all other possible trees. This mechanism is naturally viewed as a way of defining a new (...) class of tree languages. We formalize the tree language thus defined, study its expressive power, and show that the latter is beyond context freeness. While the increased expressive power offered by PCFGs helps to reduce ambiguity, we show that, in general, it cannot be decided whether a PCFG removes all ambiguities. (shrink)

This article provides an overview of a probabilistic constraints framework for thinking about language acquisition and processing. The generative approach attempts to characterize knowledge of language (i.e., competence grammar) and then asks how this knowledge is acquired and used. Our approach is performance oriented: the goal is to explain how people comprehend and produce utterances and how children acquire this skill. Use of language involves exploiting multiple probabilistic constraints over various types of linguistic and nonlinguistic information. Acquisition is (...) the process of accumulating this information, which begins in infancy. The constraint satisfaction processes that are central to language use are the same as the bootstrapping processes that provide entry to language for the child. Framing questions about acquisition in terms of models of adult performance unifies the two topics under a set of common principles and has important consequences for arguments concerning language learnability. (shrink)

Remarkable progress in the mathematics and computer science of probability has led to a revolution in the scope of probabilistic models. In particular, ‘sophisticated’ probabilistic methods apply to structured relational systems such as graphs and grammars, of immediate relevance to the cognitive sciences. This Special Issue outlines progress in this rapidly developing field, which provides a potentially unifying perspective across a wide range of domains and levels of explanation. Here, we introduce the historical and conceptual foundations of (...) the approach, explore how the approach relates to studies of explicit probabilistic reasoning, and give a brief overview of the field as it stands today. (shrink)

A probabilistic Chomsky–Schützenberger hierarchy of grammars is introduced and studied, with the aim of understanding the expressive power of generative models. We offer characterizations of the distributions definable at each level of the hierarchy, including probabilistic regular, context-free, (linear) indexed, context-sensitive, and unrestricted grammars, each corresponding to familiar probabilistic machine classes. Special attention is given to distributions on (unary notations for) positive integers. Unlike in the classical case where the "semi-linear" languages all collapse into the (...) regular languages, using analytic tools adapted from the classical setting we show there is no collapse in the probabilistic hierarchy: more distributions become definable at each level. We also address related issues such as closure under probabilistic conditioning. (shrink)

We examine the expressive power of probabilistic context free grammars (PCFGs), with a special focus on the use of probabilities as a mechanism for reducing ambiguity by filtering out unwanted parses. Probabilities in PCFGs induce an ordering relation among the set of trees that yield a given input sentence. PCFG parsers return the trees bearing the maximum probability for a given sentence, discarding all other possible trees. This mechanism is naturally viewed as a way of defining a new (...) class of tree languages. We formalize the tree language thus defined, study its expressive power, and show that the latter is beyond context freeness. While the increased expressive power offered by PCFGs helps to reduce ambiguity, we show that, in general, it cannot be decided whether a PCFG removes all ambiguities. (shrink)

More and more researchers are examining grammar acquisition from theoretical perspectives that treat it as an emergent phenomenon. In this essay, I argue that a robustly developmental perspective provides a potential explanation for some of the well-known crosslinguistic features of early child language: the process of acquisition is shaped in part by the developmental constraints embodied in von Baer’s law of development. An established model of development, the Developmental Lock, captures and elucidates the probabilistic generalizations at the heart of (...) von Baer’s law. When this model is applied to the acquisition of grammar, it predicts that grammatical achievements that are more generatively entrenched will emerge earlier in development and will be more developmentally resilient than those that are less generatively entrenched. I show that the first prediction is supported by a wealth of psycholinguistic evidence involving typically developing children and that the second prediction is supported by numerous studies involving both children who receive deficient linguistic input and children who experience various language impairments. The success of this model demonstrates the analytic potential of a developmental approach to the study of language acquisition. (shrink)

While Ç ´Ò¿ µ methods for parsing probabilistic context-free grammars (PCFGs) are well known, a tabular parsing framework for arbitrary PCFGs which allows for botton-up, topdown, and other parsing strategies, has not yet been provided. This paper presents such an algorithm, and shows its correctness and advantages over prior work. The paper finishes by bringing out the connections between the algorithm and work on hypergraphs, which permits us to extend the presented Viterbi (best parse) algorithm to an inside (...) (total probability) algorithm. (shrink)

The problems of access—retrieving linguistic structure from some mental grammar —and disambiguation—choosing among these structures to correctly parse ambiguous linguistic input—are fundamental to language understanding. The literature abounds with psychological results on lexical access, the access of idioms, syntactic rule access, parsing preferences, syntactic disambiguation, and the processing of garden‐path sentences. Unfortunately, it has been difficult to combine models which account for these results to build a general, uniform model of access and disambiguation at the lexical, idiomatic, and syntactic levels. (...) For example, psycholinguistic theories of lexical access and idiom access and parsing theories of syntactic rule access have almost no commonality in methodology or coverage of psycholinguistic data. This article presents a single probabilistic algorithm which models both the access and disambiguation of linguistic knowledge. The algorithm is based on a parallel parser which ranks constructions for access, and interpretations for disambiguation, by their conditional probability. Low‐ranked constructions and interpretations are pruned through beam‐search; this pruning accounts, among other things, for the garden‐path effect. I show that this motivated probabilistic treatment accounts for a wide variety of psycholinguistic results, arguing for a more uniform representation of linguistic knowledge and for the use of probabilistically‐enriched grammars and interpreters as models of human knowledge of and processing of language. (shrink)

One of the integral parts of Bayesian coherentism is the view that the relation of ‘being no less coherent than’ is fully determined by the probabilistic features of the sets of propositions to be ordered. In the last one and a half decades, a variety of probabilistic measures of coherence have been put forward. However, there is large disagreement as to which of these measures best captures the pre-theoretic notion of coherence. This paper contributes to the debate on (...) coherence measures by considering three classes of adequacy constraints. Various independence and dependence relations between the members of each class will be taken into account in order to reveal the ‘grammar’ of probabilistic coherence measures. Afterwards, existing proposals are examined with respect to this list of desiderata. Given that for purely mathematical reasons there can be no measure that satisfies all constraints, the grammar allows the coherentist to articulate an informed pluralist stance as regards probabilistic measures of coherence. (shrink)

Filler-gap dependency resolution is often characterized as an active process. We probed the mechanisms that determine where and why comprehenders posit gaps during incremental processing using Norwegian as our test language. First, we investigated why active filler-gap dependency resolution is suspended inside island domains like embedded questions in some languages. Processing-based accounts hold that resource limitations prevent gap-filling in embedded questions across languages, while grammar-based accounts predict that active gap-filling is only blocked in languages where embedded questions are grammatical islands. (...) In a self-paced reading study, we find that Norwegian participants exhibit filled-gap effects inside embedded questions, which are not islands in the language. The findings are consistent with grammar-based, but not processing, accounts. Second, we asked if active filler-gap processing can be understood as a special case of probabilistic ambiguity resolution within an expectation-based framework. To do so, we tested whether word-by-word surprisal values from a neural language model could predict the location and magnitude of filled-gap effects in our behavioral data. We find that surprisal accurately tracks the location of filled-gap effects but severely underestimates their magnitude. This suggests either that mechanisms above and beyond probabilistic ambiguity resolution are required to fully explain active gap-filling behavior or that surprisal values derived from long-short term memory are not good proxies for humans' incremental expectations during filler-gap resolution. (shrink)

The question of whether humans represent grammatical knowledge as a binary condition on membership in a set of well-formed sentences, or as a probabilistic property has been the subject of debate among linguists, psychologists, and cognitive scientists for many decades. Acceptability judgments present a serious problem for both classical binary and probabilistic theories of grammaticality. These judgements are gradient in nature, and so cannot be directly accommodated in a binary formal grammar. However, it is also not possible to (...) simply reduce acceptability to probability. The acceptability of a sentence is not the same as the likelihood of its occurrence, which is, in part, determined by factors like sentence length and lexical frequency. In this paper, we present the results of a set of large-scale experiments using crowd-sourced acceptability judgments that demonstrate gradience to be a pervasive feature in acceptability judgments. We then show how one can predict acceptability judgments on the basis of probability by augmenting probabilistic language models with an acceptability measure. This is a function that normalizes probability values to eliminate the confounding factors of length and lexical frequency. We describe a sequence of modeling experiments with unsupervised language models drawn from state-of-the-art machine learning methods in natural language processing. Several of these models achieve very encouraging levels of accuracy in the acceptability prediction task, as measured by the correlation between the acceptability measure scores and mean human acceptability values. We consider the relevance of these results to the debate on the nature of grammatical competence, and we argue that they support the view that linguistic knowledge can be intrinsically probabilistic. (shrink)

This paper presents a novel approach to the unsupervised learning of syntactic analyses of natural language text. Most previous work has focused on maximizing likelihood according to generative PCFG models. In contrast, we employ a simpler probabilistic model over trees based directly on constituent identity and linear context, and use an EM-like iterative procedure to induce structure. This method produces much higher quality analyses, giving the best published results on the ATIS dataset.

This paper examines feature selection for log linear models over rich constraint-based grammar (HPSG) representations by building decision trees over features in corresponding probabilistic context free grammars (PCFGs). We show that single decision trees do not make optimal use of the available information; constructed ensembles of decision trees based on different feature subspaces show signifi- cant performance gains (14% parse selection error reduction). We compare the performance of the learned PCFG grammars and log linear models over the (...) same features. (shrink)

A word-by-word human sentence processing complexity metric is presented. This metric formalizes the intuition that comprehenders have more trouble on words contributing larger amounts of information about the syntactic structure of the sentence as a whole. The formalization is in terms of the conditional entropy of grammatical continuations, given the words that have been heard so far. To calculate the predictions of this metric, Wilson and Carroll's (1954) original entropy reduction idea is extended to infinite languages. This is demonstrated with (...) a mildly context-sensitive language that includes relative clauses formed on a variety of grammatical relations across the Accessibility Hierarchy of Keenan and Comrie (1977). Predictions are derived that correlate significantly with repetition accuracy results obtained in a sentence-memory experiment (Keenan & Hawkins, 1987). (shrink)

Recent evidence suggests that probabilistic grammars may be modulated by communication mode and genre. Accordingly, the question arises how complex language users’ lectal competence is, where complexity is proportional to the extent to which choice-making processes depend on the situation of language use. Do probabilistic constraints vary when we talk to a friend compared to when we give a speech? Are differences between spoken and written language larger than those within each mode? In the present study, we (...) aim to approach these questions systematically. Guided by theorizing in cognitive (socio)linguistics and using logistic regression based on corpus materials, we analyzed the dative alternation with give (The government gives farmers money vs. The government gives money to farmers) in four broad registers of English: spoken informal, spoken formal, written informal, and written formal. Corpus analysis was supplemented with a scalar rating experiment. Results suggest that language users’ probabilistic grammars vary as a function of register. (shrink)

Minimalist grammars (MGs) and multiple context-free grammars (MCFGs) are weakly equivalent in the sense that they define the same languages, a large mildly context-sensitive class that properly includes context-free languages. But in addition, for each MG, there is an MCFG which is strongly equivalent in the sense that it defines the same language with isomorphic derivations. However, the structure-building rules of MGs but not MCFGs are defined in a way that generalizes across categories. Consequently, MGs can be exponentially (...) more succinct than their MCFG equivalents, and this difference shows in parsing models too. An incremental, top-down beam parser for MGs is defined here, sound and complete for all MGs, and hence also capable of parsing all MCFG languages. But since the parser represents its grammar transparently, the relative succinctness of MGs is again evident. Although the determinants of MG structure are narrowly and discretely defined, probabilistic influences from a much broader domain can influence even the earliest analytic steps, allowing frequency and context effects to come early and from almost anywhere, as expected in incremental models. (shrink)

The task of inducing grammar structures has received a great deal of attention. The reasons why researchers have studied are different; to use grammar induction as the first stage in building large treebanks or to make up better language models. However, grammar induction has inherent computational complexity. To overcome it, some grammar induction algorithms add new production rules incrementally. They refine the grammar while keeping their computational complexity low. In this paper, we propose a new efficient grammar induction algorithm. Although (...) our algorithm is similar to algorithms which learn a grammar incrementally, our algorithm uses the graphical EM algorithm instead of the Inside-Outside algorithm. We report results of learning experiments in terms of learning speeds. The results show that our algorithm learns a grammar in constant time regardless of the size of the grammar. Since our algorithm decreases syntactic ambiguities in each step, our algorithm reduces required time for learning. This constant-time learning considerably affects learning time for larger grammars. We also reports results of evaluation of criteria to choose nonterminals. Our algorithm refines a grammar based on a nonterminal in each step. Since there can be several criteria to decide which nonterminal is the best, we evaluate them by learning experiments. (shrink)

A number of studies report that frequency is a poor predictor of acceptability, in particular at the lower end of the frequency spectrum. Because acceptability judgments provide a substantial part of the empirical foundation of dominant linguistic traditions, understanding how acceptability relates to frequency, one of the most robust predictors of human performance, is crucial. The relation between low frequency and acceptability is investigated using corpus‐ and behavioral data on the distribution of infinitival and finite that‐complements in Polish. Polish verbs (...) exhibit substantial subordination variation and for the majority of verbs taking an infinitival complement, the that‐complement occurs with low frequency (<0.66 ipm). These low‐frequency that‐clauses, in turn, exhibit large differences in how acceptable they are to native speakers. It is argued that acceptability judgments are based on configurations of internally structured exemplars, the acceptability of which cannot reliably be assessed until sufficient evidence about the core component has accumulated. (shrink)

The acquisition of complex motor, cognitive, and social skills, like playing a musical instrument or mastering sports or a language, is generally associated with implicit skill learning . Although it is a general view that SL is most effective in childhood, and such skills are best acquired if learning starts early, this idea has rarely been tested by systematic empirical studies on the developmental pathways of SL from childhood to old age. In this paper, we challenge the view that childhood (...) and early school years are the prime time for skill learning by tracking age-related changes in performance in three different paradigms of SL. We collected data from participants between 7 and 87 years for a Serial Reaction Time Task testing the learning of motor sequences, an Artificial Grammar Learning task testing the extraction of regularities from auditory sequences, and Probabilistic Category Learning in the Weather Prediction task , a non-sequential categorization task. Results on all three tasks show that adolescence and adulthood are the most efficient periods for skill learning, since instead of becoming less and less effective with age, SL improves from childhood into adulthood and then later declines with aging. (shrink)

Exploring the concepts of Freezing: Theoretical and empirical perspectives. Theoretical advancement. Ur Shlonsky and Luigi Rizzi: Criterial Freezing in small clauses and the cartography of copular constructions -- Angel J. Gallego: Freezing Effects in a free-Merge System -- Gereon Maller: Freezing in complex prefields. Empirical domains. Norbert Corver: The Freezing points of the (Dutch) adjectival system -- Jutta M. Hartmann: Freezing in it-clefts: Movement and focus -- Josef Bayer: Criterial Freezing in the syntax of particles -- Michael S. Rochemont: Only (...) syntax -- Maria Polinsky: Freezing and phi-feature agreement: On the role of [PERSON] -- BalÃízs SurÃínyi and Gerg Turi: Freezing, Topic Opacity and Phase-based Cyclicity in Subject Islands. Interface extensions. Peter W. Culicover and Susanne Winkler: Freezing: Between grammar and processing -- Andreas Konietzko: Heavy NP shift in context: On the interaction of information structure and subextraction from shifted constituents -- Rui P. Chaves: Freezing as a probabilistic phenomenon -- Marion Jaeger: An experimental study on freezing and topicalization in English. (shrink)

A number of studies report that frequency is a poor predictor of acceptability, in particular at the lower end of the frequency spectrum. Because acceptability judgments provide a substantial part of the empirical foundation of dominant linguistic traditions, understanding how acceptability relates to frequency, one of the most robust predictors of human performance, is crucial. The relation between low frequency and acceptability is investigated using corpus- and behavioral data on the distribution of infinitival and finite that-complements in Polish. Polish verbs (...) exhibit substantial subordination variation and for the majority of verbs taking an infinitival complement, the that-complement occurs with low frequency. These low-frequency that-clauses, in turn, exhibit large differences in how acceptable they are to native speakers. It is argued that acceptability judgments are based on configurations of internally structured exemplars, the acceptability of which cannot reliably be assessed until sufficient evidence about the core component has accumulated. (shrink)

This article proposes a new model of human concept learning that provides a rational analysis of learning feature‐based concepts. This model is built upon Bayesian inference for a grammatically structured hypothesis space—a concept language of logical rules. This article compares the model predictions to human generalization judgments in several well‐known category learning experiments, and finds good agreement for both average and individual participant generalizations. This article further investigates judgments for a broad set of 7‐feature concepts—a more natural setting in several (...) ways—and again finds that the model explains human performance. (shrink)

In the Bayesian framework, a language learner should seek a grammar that explains observed data well and is also a priori probable. This paper proposes such a measure of prior probability. Indeed it develops a full statistical framework for lexicalized syntax. The learner's job is to discover the system of probabilistic transformations (often called lexical redundancy rules) that underlies the patterns of regular and irregular syntactic constructions listed in the lexicon. Specifically, the learner discovers what transformations apply in the (...) language, how often they apply, and in what contexts. It considers simpler systems of transformations to be more probable a priori. Experiments show that the learned transformations are more effective than previous statistical models at predicting the probabilities of lexical entries, especially those for which the learner had no direct evidence. (shrink)

Comparing the relative sensitivity of direct and indirect measures of learning is proposed as the best way to provide evidence for unconscious learning when both conceptual and operative definitions of awareness are lacking. This approach was first proposed by Reingold & Merikle (1988) in the context of subliminal perception. In this paper, we apply it to a choice reaction time task in which the material is generated based on a probabilistic finite-state grammar (Cleeremans, 1993). We show (1) that participants (...) progressively learn about the statistical structure of the stimulus material over training with the choice reaction time task, and (2) that they can use some of this knowledge to predict the location of the next stimulus in a subsequent “generation” task. However, detailed partial correlational analyses of the correspondence between performance during the reaction time task and the statistical structure of the training material showed that large effects remained even when controlling for explicit knowledge as assessed by the generation task. Hence we conclude (1) that at least some of the knowledge expressed through reaction time performance can not be characterized as conscious, and (2) that even when associations are found at a global level of analysis, dissociations can still be obtained when more detailed analyses are conducted. Finally, we also show that participants are limited in the depth of the contingencies they can learn about, and that these limitations are shared by the Simple Recurrent Network model of Cleeremans & McClelland (1991). (shrink)

This book scrutinizes recent work in phonological theory from the perspective of Chomskyan generative linguistics and argues that progress in the field depends on taking seriously the idea that phonology is best studied as a mental computational system derived from an innate base, phonological Universal Grammar. Two simple problems of phonological analysis provide a frame for a variety of topics throughout the book. The competence-performance distinction and markedness theory are both addressed in some detail, especially with reference to phonological acquisition. (...) Several aspects of Optimality Theory, including the use of Output-Output Correspondence, functionalist argumentation and dependence on typological justification are critiqued. The authors draw on their expertise in historical linguistics to argue that diachronic evidence is often mis-used to bolster phonological arguments, and they present a vision of the proper use of such evidence. Issues of general interest for cognitive scientists, such as whether categories are discrete and whether mental computation is probabilistic are also addressed. The book ends with concrete proposals to guide future phonological research.The breadth and depth of the discussion, ranging from details of current analyses to the philosophical underpinnings of linguistic science, is presented in a direct style with as little recourse to technical language as possible. (shrink)

Over the past decade, sequence learning has gradually become a central paradigm through which to study implicit learning. In this chapter, we start by briefly summarizing the results obtained with different variants of the sequence learning paradigm. We distinguish three subparadigms in terms of whether the stimulus material is generated either by following a fixed and repeating sequence (e.g., Nissen & Bullemer, 1987), by relying on a complex set of rules from which one can produce several alternative deterministic sequences (e.g., (...) Lewicki, Hill & Bizot, 1988; Stadler, 1989), or by following the output of a probabilistic set of rules such as instantiated by noisy finite-state grammars (Cleeremans & McClelland, 1991; Jiménez, Mendéz & Cleeremans, 1996). Next, we focus on the processes involved in sequence representation and acquisition. We suggest that the sensitivity to the sequential structure observed in the probabilistic subparadigm can only be a result of the acquisition of a representation of the statistical constraints of the material, and that this sensitivity emerges through the operation of mechanisms that are well instantiated by connectionist models such as the Simple Recurrent Network (Elman, 1990; Cleeremans, 1993b). We present new simulation work meant to explore to what extent the model can also account for specific data obtained in a paradigmatic instance of deterministic, rule-based sequence learning task: Lewicki et al. (1988)'s situation. Finally, we report on the results of an experiment that compares learning on otherwise similar deterministic and probabilistic structures, and we show that learning of both types of structures is equivalent only under conditions that maximally hinder explicit acquisition. Taken together, these simulation and experimental data lend support to the claim that implicit learning in all three sequence learning subparadigms can amount to a form of statistical sequence learning. They also suggest that distinguishing among several theories of sequence representation and acquisition may require us to analize the data in great detail. Hopefully, however, some truth can be found in such details.. (shrink)

Information‐theoretic complexity metrics, such as Surprisal (Hale, 2001; Levy, 2008) and Entropy Reduction (Hale, 2003), are linking hypotheses that bridge theorized expectations about sentences and observed processing difficulty in comprehension. These expectations can be viewed as syntactic derivations constrained by a grammar. However, this expectation‐based view is not limited to syntactic information alone. The present study combines structural and non‐structural information in unified models of word‐by‐word sentence processing difficulty. Using probabilistic minimalist grammars (Stabler, 1997), we extend expectation‐based models (...) to include frequency information about noun phrase animacy. Entropy reductions derived from these grammars faithfully reflect the asymmetry between subject and object relatives (Staub, 2010; Staub, Dillon, & Clifton, 2017), as well as the effect of animacy on the measured difficulty profile (Lowder & Gordon, 2012; Traxler, Morris, & Seely, 2002). Visualizing probability distributions on the remaining alternatives at particular parser states allows us to explore new, linguistically plausible interpretations for the observed processing asymmetries, including the way that expectations about the relativized argument influence the processing of particular types of relative clauses (Wagers & Pendleton, 2016). (shrink)

Comparing the relative sensitivity of direct and indirect measures of learning is proposed as the best way to provide evidence for unconscious learning when both conceptual and operative definitions of awareness are lacking. This approach was first proposed by Reingold & Merikle (1988) in the context of subliminal perception. In this paper, we apply it to a choice reaction time task in which the material is generated based on a probabilistic finite-state grammar (Cleeremans, 1993). We show (1) that participants (...) progressively learn about the statistical structure of the stimulus material over training with the choice reaction time task, and (2) that they can use some of this knowledge to predict the location of the next stimulus in a subsequent “generation” task. However, detailed partial correlational analyses of the correspondence between performance during the reaction time task and the statistical structure of the training material showed that large effects remained even when controlling for explicit knowledge as assessed by the generation task. Hence we conclude (1) that at least some of the knowledge expressed through reaction time performance can not be characterized as conscious, and (2) that even when associations are found at a global level of analysis, dissociations can still be obtained when more detailed analyses are conducted. Finally, we also show that participants are limited in the depth of the contingencies they can learn about, and that these limitations are shared by the Simple Recurrent Network model of Cleeremans & McClelland (1991). (shrink)

The concepts of the innateness, universality, species-specificity, and autonomy of the human language capacity have had an extreme impact on the psycholinguistic debate for over thirty years. These concepts are evaluated from several neurobiological perspectives, with an emphasis on the emergence of language and its decay due to brain lesion and progressive brain disease.Evidence of perceptuomotor homologies and preadaptations for human language in nonhuman primates suggests a gradual emergence of language during hominid evolution. Regarding ontogeny, the innate component of language (...) capacity is likely to be polygenic and shared with other developmental domains. Dissociations between verbal and nonverbal development are probably rooted in the perceptuomotor specializations of neural substrates rather than the autonomy of a grammar module. Aphasiologicaldata often assumed to suggest modular linguistic subsystems can be accounted for in terms of a neurofunctional model incorporating perceptuomotor-based regional specializationsand distributivity of representations. Thus, dissociations between grammatical functors and content words are due to different conditions of acquisition and resulting differences in neural representation. Human brains are characterized by multifactorial interindividual variability, and strict universality of functional organization is biologically unrealistic.A theoretical alternative is proposed according to which (1) linguistic specialization of brain areas is due to epigenetic and probabilistic maturational events, not to genetic ”hard-wiring,” and (2) linguistic knowledge is neurally represented in distributed cell assemblies whose topography reflects the perceptuomotor modalities involved in the acquisition and use of a given item of knowledge. (shrink)

One of the central debates in the cognitive science of language has revolved around the nature of human linguistic competence. Whether syntactic competence should be characterized by abstract hierarchical structures or reduced to surface linear strings has been actively debated, but the nature of morphological competence has been insufficiently appreciated despite the parallel question in the cognitive science literature. In this paper, in order to investigate whether morphological competence should be characterized by abstract hierarchical structures, we conducted the crowdsourced acceptability (...) judgment experiment on morphologically complex words and evaluated five computational models of morphological competence against human acceptability judgments: Character Markov Models (Character), Syllable Markov Models (Syllable), Morpheme Markov Models (Morpheme), Hidden Markov Models (HMM), and Probabilistic Context-Free Grammars (PCFG). Our psycholinguistic experimentation and computational modeling demonstrated that "morphous" computational models with morpheme units outperformed "amorphous" computational models without morpheme units and, importantly, PCFG with hierarchical structures most accurately explained human acceptability judgments on several evaluation metrics, especially for morphologically complex words with nested morphological structures. Those results strongly suggest that human morphological competence should be characterized by abstract hierarchical structures internally generated by the grammar, not reduced to surface linear strings externally attested in large corpora. (shrink)

Naturally occurring speech contains only a limited amount of complex recursive structure, and this is reflected in the empirically documented difficulties that people experience when processing such structures. We present a connectionist model of human performance in processing recursive language structures. The model is trained on simple artificial languages. We find that the qualitative performance profile of the model matches human behavior, both on the relative difficulty of center‐embedding and cross‐dependency, and between the processing of these complex recursive structures and (...) right‐branching recursive constructions. We analyze how these differences in performance are reflected in the internal representations of the model by performing discriminant analyses on these representations both before and after training. Furthermore, we show how a network trained to process recursive structures can also generate such structures in a probabilistic fashion. This work suggests a novel explanation of people's limited recursive performance, without assuming the existence of a mentally represented competence grammar allowing unbounded recursion. (shrink)

Learning theory has frequently been applied to language acquisition, but discussion has largely focused on information theoretic problems—in particular on the absence of direct negative evidence. Such arguments typically neglect the probabilistic nature of cognition and learning in general. We argue first that these arguments, and analyses based on them, suffer from a major flaw: they systematically conflate the hypothesis class and the learnable concept class. As a result, they do not allow one to draw significant conclusions about the (...) learner. Second, we claim that the real problem for language learning is the computational complexity of constructing a hypothesis from input data. Studying this problem allows for a more direct approach to the object of study—the language acquisition device—rather than the learnable class of languages, which is epiphenomenal and possibly hard to characterize. The learnability results informed by complexity studies are much more insightful. They strongly suggest that target grammars need to be objective, in the sense that the primitive elements of these grammars are based on objectively definable properties of the language itself. These considerations support the view that language acquisition proceeds primarily through data-driven learning of some form. (shrink)

Children learn their native language by exposure to their linguistic and communicative environment, but apparently without requiring that their mistakes be corrected. Such learning from “positive evidence” has been viewed as raising “logical” problems for language acquisition. In particular, without correction, how is the child to recover from conjecturing an over-general grammar, which will be consistent with any sentence that the child hears? There have been many proposals concerning how this “logical problem” can be dissolved. In this study, we review (...) recent formal results showing that the learner has sufficient data to learn successfully from positive evidence, if it favors the simplest encoding of the linguistic input. Results include the learnability of linguistic prediction, grammaticality judgments, language production, and form-meaning mappings. The simplicity approach can also be “scaled down” to analyze the learnability of specific linguistic constructions, and it is amenable to empirical testing as a framework for describing human language acquisition. (shrink)

Lexical ambiguity can be syntactic if it involves more than one grammatical category for a single word, or semantic if more than one meaning can be associated with a word. In this article we discuss the application of a Bayesian‐network model in the resolution of lexical ambiguities of both types. The network we propose comprises a parsing subnetwork, which can be constructed automatically for any context‐free grammar, and a subnetwork for semantic analysis, which, in the spirit of Fillmore's (1968) case (...) grammars, seeks to fulfill the required cases of all candidates for verb of the sentence. Solving for the highest joint probability of the variables conditioned upon the evidences to the network yields the most likely candidate with its meaning, along with its cases and respective meanings. This is achieved by fixing the values of all evidence nodes concurrently, and then performing a stochastic simulation in which the remaining nodes are updated probabilistically with a high degree of parallelism. The process of disambiguation is directed neither by the syntax nor the semantics, but rather by the interrelation between the two subnetworks. The use of a Bayesian‐network model allows us to express this interrelation between the two subnetworks and among their constituents in a rather direct and rigorous way that, in connection with the convergence properties of the stochastic simulation, reveals a very robust model. (shrink)

This paper shows how to define probability distributions over linguistically realistic syntactic structures in a way that permits us to define language learning and language comprehension as statistical problems. We demonstrate our approach using lexical‐functional grammar (LFG), but our approach generalizes to virtually any linguistic theory. Our probabilistic models are maximum entropy models. In this paper we concentrate on statistical inference procedures for learning the parameters that define these probability distributions. We point out some of the practical problems that (...) make straightforward ways of estimating these distributions infeasible, and develop a “pseudo‐likelihood” estimation procedure that overcomes some of these problems. This method raises interesting questions concerning the nature of the data available to a language learner and the modularity of language learning and processing. (shrink)

The ability to extract patterns from sensory input across time and space is thought to underlie the development and acquisition of language and literacy skills, particularly the subdomains marked by the learning of probabilistic knowledge. Thus, impairments in procedural learning are hypothesized to underlie neurodevelopmental disorders, such as dyslexia and developmental language disorder. In the present meta‐analysis, comprising 2396 participants from 39 independent studies, the continuous relationship between language, literacy, and procedural learning on the Serial Reaction Time task (SRTT) (...) was assessed across children and adults with typical development (TD), dyslexia, and Developmental Language Disorder (DLD). Despite a significant, but very small, relationship between procedural learning and overall language and literacy measures, this pattern was not observed at the group‐level when examining TD, dyslexic, and DLD groups separately. Based on the procedural/declarative model, a positive relationship was expected between procedural learning and language and literacy measures for the typically developing group; however, no such relationship was observed. This was also the case for the disordered groups (ps >.05). Also counter to expectations, the magnitude of the relationship between procedural learning and grammar and phonology did not differ between TD and DLD (ps >.05), nor between the TD and dyslexic group on reading, spelling, and phonology (ps >.05). While lending little support to the procedural/declarative model, we consider that these results may be the consequence of poor psychometric properties of the SRTT as a measure of procedural learning. (shrink)

In recent decades, the analysis of phraseology has made use of the exploration of large corpora as a source of quantitative information about language. This paper intends to present the main lines of work in progress based on this empirical approach to linguistic analysis. In particular, we focus our attention on some problems relating to the morpho-syntactic annotation of corpora. The CORIS/CODIS corpus of contemporary written Italian, developed at CILTA – University of Bologna (Rossini Favretti 2000; Rossini Favretti, Tamburini, De (...) Santis in press), is a synchronic 100-million-word corpus and is being lemmatised and annotated with part-of-speech (POS) tags, in order to increase the quantity of information and improve data retrieval procedures (Tamburini 2000). The aim of POS tagging is to assign each lexical unit to the appropriate word class. Usually the set of tags is pre-established by the linguist, who uses his/her competence to identify the different word classes. The very first experiments we made revealed how the traditional part-of-speech distinctions in Italian (generally based on morphological and semantic criteria) are often inadequate to represent the syntactic features of words in context. It is worth noting that the uncertainties in categorisation contained in Italian grammars and dictionaries reflect a growing difficulty as they move from fundamental linguistic classes, such as nouns and verbs, to more complex classes, such as adverbs, pronouns, prepositions and conjunctions. This latter class, that groups together elements traditionally used to express connections between sentences, appears inadequate when describing cohesive relations in Italian. This phenomenon actually seems to involve other elements traditionally assigned to different classes, such as adverbs, pronouns and interjections. Recent studies proposed the class of ‘connectives’, grouping all words that, apart from their traditional word class, have the function of connecting phrases and contributing to textual cohesion. From this point of view, conjunctions can be considered as part of phrasal connectives, that can in turn be included in the wider category of textual connectives. The aim of this study is to identify elements that can be included in the class of phrasal connectives, using quantitative methods. According to Shannon and Weaver’s (1949) observation that words are linked by dependent probabilities, corroborated by Halliday’s (1991) argument that the grammatical “system” (in Firth’s sense of the term) is essentially probabilistic, quantitative data are introduced in order to provide evidence of relative frequencies. Section 2 presents a description of word-class categorisation from the point of view of grammars and dictionaries arguing that the traditional category of conjunctions is inadequate for capturing the notion of phrasal connective. Section 3 examines the notion of ‘connective’ and suggests a truth-function interpretation of connective behaviour. Section 4 describes the quantitative methods proposed for analysing the distributional properties of lexical units, and section 5 comments on the results obtained by applying such methods drawing some provisional conclusions. (shrink)

In lieu of an abstract, here is a brief excerpt of the content:BOOK REVIEWS 533 Grammar of Assent. Yet whereas Reid had urged a fundamental agree· ment on first principles on the intuitive basis of common sense, Newman thought such principles were discovered inductively and that there might he much disagreement. It was the disagreement itself that led to the need for a better understanding of the reasoning process. In place of common sense, Newman appealed to the illative sense. In (...) l:his regard, Newman directly rejeots Locke's distinction between demon· stration and probability. Various kinds of probabilistic reasoning for Newman can lead to a certainty beyond reasonable doubt. Ferreira claims that in this regard Newman is following a strategy not unlike l:hat of Reid. Both Reid's common sense and Newman's illative sense are natural. However, the former can discover only self-evident 'truths while the latter is part of the process of reasoning itself. Ferreira pro· vides a very full discussion of the manner in which these outlooks led Newman into the tradition of a naturalistic response to scepticism. Both philosophers and intellectual historians will find Professor Ferreira's volume useful and informative. In a very sprightly man· ner she has explored a tradition of British intellectual life that often has remained ignored. She has displayed very considerable daring in attempting to cover two centuries of thought. The most valuable sec· tions are no doubt those on Reid and Newman where she has care· fully illuminated a major intelleotual path not.taken hy most late nine· teenth· and twentieth-century British philosophers hut which exerted very considerable influence during the first three-quarters of rthe nine· teenth century. The volume also prepares the way for intellectual his· torians to examine what were the social and struotural reasons within British intellectual life for these particular anti-sceptical strategies. FRANK M. TURNER Yale University New Haven, Connecticut The Later Wittgenstein: The Emergence of a New Philosophical Meth· od. By S. STEPHEN HILMY. Oxford: Basil Blackwell, 1987. Pp. viii+ 340. This is a hook of extraordinary scholarly density. Its 226 pages of text are complemented hy 94 pages of notes, 6 pages of bibliography, and 14 pages of indices. A heavy texture of relentlessly documented argument, Hilmy's hook is neither for philosophical novices nor for!:hose whose interest in Wittgenstein is merely moderate. To profit from 534 BOOK REVIEWS this hook, the reader must share the intense concern that led the author to plough through Wittgenstein's vast Nachlass. But readers must also he willing to contend with a style of writing that is often forbiddingly and needlessly convoluted, and with a tone that is sometimes snide. More will he said about these problems below, after a discussion of the hook's purposes. Hilmy believes Wittgenstein scholarship to he in a sorry state. The problem stems, he thinks, from blunders committed hy the literary executors. He writes, " The unhappy state of Wittgenstein scholarship is in large part due to the fragmented and ahistorical character of the potpourri of published remarks wirth which scholars have been work· ing" (viii). Indeed, his confidence in the published materials is so low that he often refers to them as Wittgenstein's "works" (in quotation marks) to signal his disdain for the editors' selections and arrange· ments. The scope of this volume is intentionally limited; little or no assess· ment of Wittgenstein's conclusions is offered. But Hilmy does claim to he taking a necessary first step which, he says, "much of the volumin· ous literature... has dismally failed to take" (3); namely, an examination of.the historical development of Wittgenstein's later way of thinking (Denkweise) as chronicled in the Nachlass. His appeal to the manuscript material is based on his belief that rthe "conglomerated fragments" (9) in the published works are best understood in their original contexts and in light of later contexts into which Wittgenstein placed them in the process of revision. Much of the stylistic character of Hilmy's hook stems from.the necessarily laborious nature of trac· ing these origins and transpositions, and from his sense of getting Witt· genstein right for the first time. After sorting through some preliminary issues concerning Wittgenstein 's compositional style, Hilmy produces an... (shrink)

Parkinson’s Disease can be treated with the use of microelectrode recording and stimulation. This paper presents a data stream classifier that analyses raw data from micro-electrodes and decides whether the measurements were taken from the subthalamic nucleus (STN) or not. The novelty of the proposed approach is based on the fact that distances based on raw data are used. Two distances are investigated in this paper, i.e. Normalized Compression Distance (NCD) and Lempel-Ziv Jaccard Distance (LZJD). No new features needed to (...) be extracted due to the fact that in the case of high-dimensional data the process is extremely time-consuming. The k-nearest neighbour (k-NN) was chosen as the classifier due to its simplicity, which is essential in data stream classification. Results obtained from classifiers based on k-NN: k-NN, k-NN were compared with Probabilistic Approximate Window (k-NN with PAW); k-NN with Probabilistic Approximate Window and Adaptive Windowing (k-NN with PAW and ADWIN); and Self Adjusting Memory k-NN (SAM k-NN), which use the proposed distances, with the performance of the same classifiers but using standard Euclidean distance. Prequential accuracy was chosen as the performance measure. The results of the experiments performed with the described approach are in most cases better, i.e. the performance measures for kNN classifiers that use NCD and LZJD distances are better by up to 8.5 per cent and 14 per cent, respectively. Moreover, the proposed approach performs better when compared with other stream classification algorithms, i.e. Hoeffding Tree, Naive Bayes, and Leveraging Bagging. In the discussed case, an improvement of classification rate of up to 17.9 per cent when using Lempel-Ziv Jaccard Distance instead of the Euclidean was noted. (shrink)

Different approaches analyze the strength of a natural language argument in different ways. This paper contrasts the dialectical, structural, probabilistic (or Bayesian), computational, and empirical approaches by exemplarily applying them to a single argumentative text (Epicureans on Squandering Life; Aikin & Talisse, 2019). Rather than pitching these approaches against one another, our main goal is to show the room for fruitful interaction. Our focus is on a dialectical analysis of the squandering argument as an argumentative response that voids an (...) interlocutor’s right to assertion. This analysis addresses the pragmatic dimensions of arguing and implies an argument structure that is consistent with empirical evidence of perceived argument strength. Results show that the squandering argument can be evaluated as a (non-fallacious) ad hominem argument, which however is not necessarily stronger than possible arguments attacking it. (shrink)