In this important and pioneering book Frederick Newmeyer takes on the question of language variety. He considers why some language types are impossible and why some grammatical features are more common than others. The task of trying to explain typological variation among languages has been mainly undertaken by functionally-oriented linguists. Generative grammarians entering the field of typology in the 1980s put forward the idea that cross-linguistic differences could be explained by linguistic parameters within Universal Grammar, whose operation (...) might vary from language to language. Unfortunately, this way of looking at variation turned out to be much less successful than had been hoped for. Professor Newmeyer's alternative to parameters combines leading ideas from functionalist and formalist approaches which in the past have been considered incompatible. He throws fresh light on language typology and variation, and provides new insights into the principles of Universal Grammar. The book is written in a clear, readable style and will be readily understood by anyone with a couple of years' study of linguistics. It will interest a wide range of scholars and students of language, including typologists, historical linguists, and theorists of every shade. (shrink)

This paper presents a novel approach to the unsupervised learning of syntactic analyses of natural language text. Most previous work has focused on maximizing likelihood according to generative PCFG models. In contrast, we employ a simpler probabilistic model over trees based directly on constituent identity and linear context, and use an EM-like iterative procedure to induce structure. This method produces much higher quality analyses, giving the best published results on the ATIS dataset.

We introduce a set of biologically and computationally motivated design choices for modeling the learning of language, or of other types of sequential, hierarchically structured experience and behavior, and describe an implemented system that conforms to these choices and is capable of unsupervised learning from raw natural-language corpora. Given a stream of linguistic input, our model incrementally learns a grammar that captures its statistical patterns, which can then be used to parse or generate new data. The grammar constructed (...) in this manner takes the form of a directed weighted graph, whose nodes are recursively (hierarchically) defined patterns over the elements of the input stream. We evaluated the model in seventeen experiments, grouped into five studies, which examined, respectively, (a) the generative ability of grammar learned from a corpus of natural language, (b) the characteristics of the learned representation, (c) sequence segmentation and chunking, (d) artificial grammar learning, and (e) certain types of structure dependence. The model's performance largely vindicates our design choices, suggesting that progress in modeling language acquisition can be made on a broad front—ranging from issues of generativity to the replication of human experimental findings—by bringing biological and computational considerations, as well as lessons from prior efforts, to bear on the modeling approach. (shrink)

The theory of language must predict the possible thought—signal (or meaning—sound or sign) pairings of a language. We argue for a Meaning First architecture of language where a thought structure is generated first. The thought structure is then realized using language to communicate the thought, to memorize it, or perhaps with another purpose. Our view contrasts with the T-model architecture of mainstream generative grammar, according to which distinct phrase-structural representations—Phonetic Form (PF) for articulation, Logical Form (...) (LF) for interpretation—are generated within the grammar. At the same time, our view differs from early transformational grammar and generative semantics: We view the relationship between the thought structure and the corresponding signal as one of compression. We specify a formal sketch of compression as a choice between multiple possible pronounciations balancing the desire to transmit information against the effort of pronounciation. The Meaning First architecture allows a greater degree of independence between thought structures and the linguistic signal. We present three arguments favoring this type of independence. First we argue that scopal properties can be better explained if we only compare thought structures independent of the their realization as a sentence. Secondly, we argue that Meaning First architecture allows contentful late insertion, an idea that has been argued for in Distributed Morphology already, but as we argue is also motivated by the division of the logical and socio-emotive meaning content of language. Finally, we show that only the Meaning First architecture provides a satisfying account of the mixing of multiple languages by multilingual speakers, especially for cases of simultaneous articulation across two modalities in bimodal speakers. Our view of the structure of grammar leads to a reassessment of priorities in linguistic analyses: while current mainstream work is often focused on establishing one-to-one relationships between concepts and morphemes, our view makes it plausible that primitive concepts are frequently marked indirectly or unpronounced entirely. Our view therefore assigns great value to the understanding of logical primitives and of compression. (shrink)

We present a generative distributional model for the unsupervised induction of natural language syntax which explicitly models constituent yields and contexts. Parameter search with EM produces higher quality analyses than previously exhibited by unsupervised systems, giving the best published unsupervised parsing results on the ATIS corpus. Experiments on Penn treebank sentences of comparable length show an even higher F1 of 71% on nontrivial brackets. We compare distributionally induced and actual part-of-speech tags as input data, and examine extensions to (...) the basic model. We discuss errors made by the system, compare the system to previous models, and discuss upper bounds, lower bounds, and stability for this task. (shrink)

Predictive Processing (PP) is an increasingly influential neurocognitive-computational framework. PP research has so far focused predominantly on lower level perceptual, motor, and various psychological phenomena. But PP seems to face a “scale-up challenge”: How can it be extended to conceptual thought, language, and other higher cognitive competencies? Compositionality, arguably a central feature of conceptual thought, cannot easily be accounted for in PP because it is not couched in terms of classical symbol processing. I argue, using the example of (...) class='Hi'>language, that there is no strong reason to think that PP cannot be scaled up to higher cognition. I suggest that the tacitly assumed common-sense conception of language as Generative Grammar (“folk linguistics”) and its notion of composition leads to the scale-up concerns. Fodor’s Language of Thought Hypothesis (LOTH) plays the role of a cognitive computational paradigm for folk linguistics. Therefore, we do not take LOTH as facing problems with higher cognition, at least with regard to compositionality. But PP can plausibly play the role of a cognitive-computational paradigm for an alternative conception of language, namely Construction Grammar. If Construction Grammar is a plausible alternative to folk linguistics, then PP is not in a worse position than LOTH. (shrink)

Like Einstein's general theory of relativity is concerned with explaining the basics of an observable experience – i.e., gravity – most people take for granted that Chomsky's theory of generative grammar (GG) is concerned with the basic nature of language. This chapter highlights a mere subset of central constructs in GG, showing how they have featured prominently and thus shaped formal linguistic studies in multilingualism. Because multilingualism includes a wide range of nonmonolingual populations, the constructs are divided across (...) child bilingualism and adult third language for greater coverage. In the case of the former, the chapter examines how poverty of the stimulus has been investigated. Using the nascent field of L3/Ln acquisition as the backdrop, it discusses how the GG constructs of I‐language versus E‐language sit at the core of debates regarding the very notion of what linguistic transfer and mental representations should be taken to be. (shrink)

This article describes an approach to connectionist language research that relies on the development of grammar formalisms rather than computer models. From formulations of the fundamental theoretical commitments of connectionism and of generative grammar, it is argued that these two paradigms are mutually compatible. Integrating the basic assumptions of the paradigms results in formal theories of grammar that centrally incorporate a certain degree of connectionist computation. Two such grammar formalisms—Harmonic Grammar and Optimality Theory —are briefly introduced to illustrate (...) grammar‐based approaches to connectionist language research. The strengths and weaknesses of grammar‐based research and more traditional model‐based research are argued to be complementary, suggesting a significant role for both strategies in the spectrum of connectionist language research. (shrink)

A study of the contribution semantics makes to the syntactic patterns of English: an intepretive theory of grammar.

In this paper, I address the issue of scientific modelling in contemporary linguistics, focusing on the generative tradition. In so doing, I identify two common varieties of linguistic idealisation, which I call determination and isolation respectively. I argue that these distinct types of idealisation can both be described within the remit of Weisberg’s :639–659, 2007) minimalist idealisation strategy in the sciences. Following a line set by Blutner :27–35, 2011), I propose this minimalist idealisation analysis for a broad construal of (...) the generative linguistic programme and thus cite examples from a wide range of linguistic frameworks including early generative syntax, Minimalism, the parallel architecture and optimality theory. Lastly, I claim that from a modelling perspective, the dynamic turn in syntax can be explained as a continuation, as opposed to a marked shift, of the generative modelling paradigm. Seen in this light, my proposal is an even broader construal of the generative tradition, along scientific modelling lines. Thus, I offer a lens through which to appreciate the scientific contribution of generative grammar, amid an increased resistance to some of its core theoretical posits, in terms of a brand of structural realism in the philosophy of science and specifically scientific modelling. (shrink)

Various restrictions on transformational grammars have been investigated in order to reduce their generative power from recursively enumerable languages to recursive languages.It will be shown that any restriction on transformational grammars defining a recursively enumerable subset of the set of all transformational grammars, is either too weak (in the sense that there does not exist a general decision procedure for all languages generated under such a restriction) or too strong (in the sense that there exists a recursive language (...) that cannot be generated by any transformational grammar thus restricted). In addition, some related problems will be discussed. (shrink)

Three long papers are collected here which constitute Chomsky’s major theoretical work on syntax and semantics subsequent to the "standard theory" of Aspects of the Theory of Syntax. Since 1965, transformational-generative linguists have suggested various changes in "standard theory," centering on the relationship between the syntactic and semantic components in natural language grammars. In these papers Chomsky explains several specific problems that require the extension of standard theory and he criticizes the proposals and arguments of the generative (...) semanticists, attempting to show that many of the specific features of language that are claimed to support the generativist position are as well or better explained by extended standard theory, and emphasizing that most of the generativist’s proposals turn out to be mere "notational variants," evidentially indistinguishable from extended standard theory. Aside from appealing to particular data favoring his "lexicalist" defense of syntactic deep structure, Chomsky makes a Popperean appeal that his position is preferable because it is more constraining and more falsifiable. (shrink)

A generative grammar for a language L generates one or more syntactic structures for each sentence of L and interprets those structures both phonologically and semantically. A widely accepted assumption in generative linguistics dating from the mid-60s, the Generative Grammar Hypothesis, is that the ability of a speaker to understand sentences of her language requires her to have tacit knowledge of a generative grammar of it, and the task of linguistic semantics in those early (...) days was taken to be that of specifying the form that the semantic component of a generative grammar must take. Then in the 70s linguistic semantics took a curious turn. Without rejecting GGH, linguists turned away from the task of characterizing the semantic component of a generative grammar to pursue instead the Montague-inspired project of providing for natural languages the same kind of model-theoretic semantics that logicians devise for the artificial languages of formal systems of logic, and “formal semantics” continues to dominate semantics in linguistics. This essay argues that the sort of compositional meaning theory that would verify GGH would not only be quite different from the theories formal semanticists construct, but would be a more fundamental theory that supersedes those theories in that it would explain why they are true when they are true, but their truth wouldn’t explain its truth. Formal semantics has undoubtedly made important contributions to our understanding of such phenomena as anaphora and quantification, but semantics in linguistics is supposed to be the study of meaning. This means that the formal semanticist can’t be unconcerned that the kind of semantic theory for a natural language that interests her has no place in a theory of linguistic competence; for if GGH is correct, then the more fundamental semantic theory is the compositional meaning theory that is the semantic component of the internally represented generative grammar, and if that is so, then linguistic semantics has so far ignored what really ought to be its primary concern. (shrink)

The goal of this study is to reintegrate the theory of generative grammar into the cognitive sciences. Generative grammar was right to focus on the child's acquisition of language as its central problem, leading to the hypothesis of an innate Universal Grammar. However, generative grammar was mistaken in assuming that the syntactic component is the sole course of combinatoriality, and that everything else is “interpretive.” The proper approach is a parallel architecture, in which phonology, syntax, and (...) semantics are autonomous generative systems linked by interface components. The parallel architecture leads to an integration within linguistics, and to a far better integration with the rest of cognitive neuroscience. It fits naturally into the larger architecture of the mind/brain and permits a properly mentalistic theory of semantics. It results in a view of linguistic performance in which the rules of grammar are directly involved in processing. Finally, it leads to a natural account of the incremental evolution of the language capacity. Key Words: evolution of language; generative grammar; parallel architecture; semantics; syntax. (shrink)

Harmonic Grammar (HG) and Optimality Theory (OT) are closely related formal frameworks for the study of language. In both, the structure of a given language is determined by the relative strengths of a set of constraints. They differ in how these strengths are represented: as numerical weights (HG) or as ranks (OT). Weighted constraints have advantages for the construction of accounts of language learning and other cognitive processes, partly because they allow for the adaptation of connectionist and (...) statistical models. HG has been little studied in generative linguistics, however, largely due to influential claims that weighted constraints make incorrect predictions about the typology of natural languages, predictions that are not shared by the more popular OT. This paper makes the case that HG is in fact a promising framework for typological research, and reviews and extends the existing arguments for weighted over ranked constraints. (shrink)

A familiar argument goes as follows: natural languages have infinitely many sentences, finite representation of infinite sets requires recursion; therefore any adequate account of linguistic competence will require some kind of recursive device. The first part of this paper argues that this argument is not convincing. The second part argues that it was not the original reason recursive devices were introduced into generative linguistics. The real basis for the use of recursive devices stems from a deeper philosophical concern; a (...) grammar that functions merely as a metalanguage would not be explanatorily adequate as it would merely push the problem of explaining linguistic competence back to another level. The paper traces this concern from Zellig Harris and Chomsky’s early work in generative linguistics and presents some implications. (shrink)

Name der Zeitschrift: Cognitive Linguistics Jahrgang: 26 Heft: 4 Seiten: 613-632.

The generative grammar approach to language seeks a fully explicit account of the modular systems of knowledge (competence) that underlies the human language capacity. Similarly, the relevance-theoretic approach to pragmatics attempts an explicit characterisation of the sub-personal systems involved in utterance interpretation. As an on-line performance system, however, it is subject to certain additional constraints; this is demonstrated by the way in which matters of computational (processing effort) economy are currently employed in the two types of theory. (...) A sub-module of “discourse competence” is shown to be compatible with and complementary to the wider system of pragmatic processes. (shrink)

A generative grammar for a language L generates one or more syntactic structures for each sentence of L and interprets those structures both phonologically and semantically. A widely accepted assumption in generative linguistics dating from the mid-60s, the Generative Grammar Hypothesis , is that the ability of a speaker to understand sentences of her language requires her to have tacit knowledge of a generative grammar of it, and the task of linguistic semantics in those (...) early days was taken to be that of specifying the form that the semantic component of a generative grammar must take. Then in the 70s linguistic semantics took a curious turn. Without rejecting GGH, linguists turned away from the task of characterizing the semantic component of a generative grammar to pursue instead the Montague-inspired project of providing for natural languages the same kind of model-theoretic semantics that logicians devise for the artificial languages of formal systems of logic, and “formal semantics” continues to dominate semantics in linguistics. This essay argues that the sort of compositional meaning theory that would verify GGH would not only be quite different from the theories formal semanticists construct, but would be a more fundamental theory that supersedes those theories in that it would explain why they are true when they are true, but their truth wouldn’t explain its truth. Formal semantics has undoubtedly made important contributions to our understanding of such phenomena as anaphora and quantification, but semantics in linguistics is supposed to be the study of meaning. This means that the formal semanticist can’t be unconcerned that the kind of semantic theory for a natural language that interests her has no place in a theory of linguistic competence; for if GGH is correct, then the more fundamental semantic theory is the compositional meaning theory that is the semantic component of the internally represented generative grammar, and if that is so, then linguistic semantics has so far ignored what really ought to be its primary concern. (shrink)

Although there is a broad consensus that both the procedural and declarative memory systems play a crucial role in language learning, use, and knowledge, the mapping between linguistic types and memory structures remains underspecified: by default, a dual-route mapping of language systems to memory systems is assumed, with declarative memory handling idiosyncratic lexical knowledge and procedural memory handling rule-governed knowledge of grammar.We experimentally contrast the processing of morphology (case and aspect), syntax (subordination), and lexical semantics (collocations) in a (...) healthy L1 population of Polish, a language rich in form distinctions. We study the processing of these four types under two conditions: a single task condition in which the grammaticality of stimuli was judged and a concurrent task condition in which grammaticality judgments were combined with a digit span task. Dividing attention impedes access to declarative memory while leaving procedural memory unaffected and hence constitutes a test that dissociates which types of linguistic information each long-term memory construct subserves.Our findings confirm the existence of a distinction between lexicon and grammar as a generative, dual-route model would predict, but the distinction is graded, as usage-based models assume: the hypothesized grammar–lexicon opposition appears as a continuum on which grammatical phenomena can be placed as being more or less “ruly” or “idiosyncratic.” However, usage-based models, too, need adjusting as not all types of linguistic knowledge are proceduralized to the same extent. This move away from a simple dichotomy fundamentally changes how we think about memory for language, and hence how we design and interpret behavioral and neuroimaging studies that probe into the nature of language cognition. (shrink)

The target article advanced the null, unified and widely misinterpreted generative hypothesis regarding second language (L2) acquisition. Postulating that UG (Universal Grammar) constrains L2 knowledge growth does not entail identical developmental trajectories for L2 and first language (LI) acquisition; nor does it preclude a role for the L1. In embracing this hypothesis, we maintain a distinction between competence and performance. Those who conflate the two repeat fundamental and by no means unprecedented misconstruals of the generative enterprise, (...) and more specifically, of the empirical content of the null hypothesis regarding L2 linguistic knowledge growth. We hope to have identified certain common goals, the adoption of which might constitute a firm foundation for continued productive interdisciplinary development of contemporary theoretical and experimental L2 acquisition research. (shrink)

A remarkable philosophical affinity may be observed between the intuitionistic conception of mathematics and the transformational generative approach to the study of language: both disciplines profess a mentalistic ontology, both posit an idealized subject, and both insist on their autonomy with respect to other disciplines. This philosophical parallel is formalized in terms of a generalization of the intuitionistic notion of creative subject; resulting are the foundations of a unified theory of mental acts based on intuitionistic logic — capturing, (...) inter alia, similarities between proof acts and speech acts. As an application of the theory, it is then shown how the notion of mental act may provide for an insightful formalization of various hypotheses pertaining to the linguistic dependence or relativity of mathematics. (shrink)

Recent defenders of the cognitive significance of religious language have had to face opponents from two directions; from those who demand that religious language be capable of some form of empirical verification and from those who demand that for religious language to be meaningful it must be capable of being understood in ordinary language. Apologists who have taken the first challenge seriously have strained to show that religious statements can be verified by ‘religious experience’, or by (...) an ‘odd discernment’ or by an ‘eschatological verification’. Each of these responses raises further problems for the defender of religion, but in general they all are subject to their failure to provide an adequate criterion or standard by which they could be inter-subjectively tested. Facing in the other direction, theological apologists have attempted to justify religious expressions by showing that they could be subsumed under special categories of ordinary language; namely those of ‘convictional language’ and ‘performative language’. Although these defenders have shown that religious language has an ‘ordinary usage’, they have not shown that the cognitive elements of this usage can be reduced to the ‘use of ordinary language’. (shrink)

Language sciences have long maintained a close and supposedly necessary coupling between the infinite productivity of the human language faculty and recursive grammars. Because of the formal equivalence between recursion and non-recursive iteration; recursion, in the technical sense, is never a necessary component of a generative grammar. Contrary to some assertions this equivalence extends to both center-embedded relative clauses and hierarchical parse trees. Inspection of language usage suggests that recursive rule components in fact contribute very little, (...) and likely nothing significant, to linguistic creativity. Further than this, if the productivity of human language is considered as not rigidly bound, but not infinite, then the need for any sort of iteration in generative grammars vanishes and can be replaced with a Non-iterative Explicit Alternatives Rule grammar. The knock-on effects of dispensing with recursive grammar components are: that language diversity can simply be based on rule and lexicon combinatorics with no potentially infinite dimensions derived from recursive or iterative components of rules; the oddity of a vast, multiply-infinite competence set of ‘grammatical but unacceptable’ productions is gone; and the development of a language faculty based on rules that eschew iterative rule components avoids any need for explaining ‘special’ mechanisms. On the broader front of searching for the mechanisms of mind, our analysis can be similarly applied to the proposals for a recursive basis for mind as an explanation for humanity’s great leap forward. (shrink)

More and more researchers are examining grammar acquisition from theoretical perspectives that treat it as an emergent phenomenon. In this essay, I argue that a robustly developmental perspective provides a potential explanation for some of the well-known crosslinguistic features of early child language: the process of acquisition is shaped in part by the developmental constraints embodied in von Baer’s law of development. An established model of development, the Developmental Lock, captures and elucidates the probabilistic generalizations at the heart of (...) von Baer’s law. When this model is applied to the acquisition of grammar, it predicts that grammatical achievements that are more generatively entrenched will emerge earlier in development and will be more developmentally resilient than those that are less generatively entrenched. I show that the first prediction is supported by a wealth of psycholinguistic evidence involving typically developing children and that the second prediction is supported by numerous studies involving both children who receive deficient linguistic input and children who experience various language impairments. The success of this model demonstrates the analytic potential of a developmental approach to the study of language acquisition. (shrink)

Jackendoff's Foundations of Language: Brain, Meaning, Grammar, Evolution attempts to reconnect generative linguistics to the rest of cognitive science. However, by minimally acknowledging decades of work in cognitive linguistics, treating dynamical systems approaches somewhat dismissively, and clinging to certain fundamental dogma while revising others, he clearly risks satisfying no one by almost pleasing everyone.

The approach to generative grammar originating with Chomsky (1957) has been enormously successful within linguistics. Seeing such success, one wonders whether a similar approach might help us understand other human domains besides language. One such domain is morality. Could there be universal generative moral grammar? More specifically, might it be useful to moral theory to develop an explicit generative account of parts of particular moralities in the way it has proved useful to linguistics to produce (...) class='Hi'>generative grammars for parts of particular languages? Should moral theorists attempt to develop a theory of moral universals that is analogous to the theory of universal grammar in linguistics? Can moral theorists develop a “principles and parameters” account of possible moralities inspired by the principles and parameters approach to language in current linguistics? Could there be a “minimalist” program for moral theory inspired by the minimalist program in linguistics? In this chapter we offer a preliminary account of some analogies, focusing on clarifying issues, making distinctions, and considering how—in a general way—such analogies might yield a fruitful research program for moral theory. There are two main parts to our discussion, one focusing on an analogy between generative grammar and moral theory, the other focusing on analogies between universal grammar and theories of moral universals. In the first part, we say a little about the background and say how we are going to understand morality and moral theory. We describe certain aspects of generative grammar and how claims about generative grammars are tested, allowing for a distinction between “competence” and “performance”. We then try to say what a corresponding “generative moral grammar” would be and how it would be tested. We next discuss a number of objections to the analogy between moral theory and generative grammar and indicate possible responses. In the second part, we discuss certain universal constraints on grammars and consider whether there might be similar constraints on moralities. Then we discuss how linguists describe core aspects of languages in terms of principles and parameters and consider what aspects of moralities might be described in similar terms. After that we make some brief remarks about minimalism. (shrink)

Unification grammars are widely accepted as an expressive means for describing the structure of natural languages. In general, the recognition problem is undecidable for unification grammars. Even with restricted variants of the formalism, off-line parsable grammars, the problem is computationally hard. We present two natural constraints on unification grammars which limit their expressivity and allow for efficient processing. We first show that non-reentrant unification grammars generate exactly the class of context-free languages. We then relax the constraint and show that one-reentrant (...) unification grammars generate exactly the class of mildly context-sensitive languages. We thus relate the commonly used and linguistically motivated formalism of unification grammars to more restricted, computationally tractable classes of languages. (shrink)

This Perspective traces the evolution of certain central notions in the theory of Generative Grammar (GG). The founding documents of the field suggested a relation between the grammar, construed as recursively enumerating an infinite set of sentences, and the idealized native speaker that was essentially equivalent to the relation between a formal language (a set of well-formed formulas) and an automaton that recognizes strings as belonging to the language or not. But this early view was later abandoned, (...) when the focus of the field shifted to the grammar's strong generative capacity as recursive generation of hierarchically structured objects as opposed to strings. The grammar is now no longer seen as specifying a set of well-formed expressions and in fact necessarily constructs expressions of any degree of intuitive “acceptability.” The field of GG, however, has not sufficiently acknowledged the significance of this shift in perspective, as evidenced by the fact that (informal and experimentally-controlled) observations about string acceptability continue to be treated as bona fide data and generalizations for the theory of GG. The focus on strong generative capacity, it is argued, requires a new discussion of what constitutes valid empirical evidence for GG beyond observations pertaining to weak generation. (shrink)

In the generative tradition, the language faculty has been shrinking—perhaps to include only the mechanism of recursion. This paper argues that even this view of the language faculty is too expansive. We first argue that a language faculty is difficult to reconcile with evolutionary considerations. We then focus on recursion as a detailed case study, arguing that our ability to process recursive structure does not rely on recursion as a property of the grammar, but instead emerges (...) gradually by piggybacking on domain-general sequence learning abilities. Evidence from genetics, comparative work on non-human primates, and cognitive neuroscience suggests that humans have evolved complex sequence learning skills, which were subsequently pressed into service to accommodate language. Constraints on sequence learning therefore have played an important role in shaping the cultural evolution of linguistic structure, including our limited abilities for processing recursive structure. Finally, we re-evaluate some of the key considerations that have often been taken to require the postulation of a language faculty. (shrink)

A probabilistic Chomsky–Schützenberger hierarchy of grammars is introduced and studied, with the aim of understanding the expressive power of generative models. We offer characterizations of the distributions definable at each level of the hierarchy, including probabilistic regular, context-free, (linear) indexed, context-sensitive, and unrestricted grammars, each corresponding to familiar probabilistic machine classes. Special attention is given to distributions on (unary notations for) positive integers. Unlike in the classical case where the "semi-linear" languages all collapse into the regular languages, using analytic (...) tools adapted from the classical setting we show there is no collapse in the probabilistic hierarchy: more distributions become definable at each level. We also address related issues such as closure under probabilistic conditioning. (shrink)