The transformers that drive chatbots and other AI systems constitute large language models (LLMs). These are currently the focus of a lively discussion in both the scientific literature and the popular media. This discussion ranges from hyperbolic claims that attribute general intelligence and sentience to LLMs, to the skeptical view that these devices are no more than “stochastic parrots”. I present an overview of some of the weak arguments that have been presented against LLMs, and I consider several of the (...) more compelling criticisms of these devices. The former significantly underestimate the capacity of transformers to achieve subtle inductive inferences required for high levels of performance on complex, cognitively significant tasks. In some instances, these arguments misconstrue the nature of deep learning. The latter criticisms identify significant limitations in the way in which transformers learn and represent patterns in data. They also point out important differences between the procedures through which deep neural networks and humans acquire knowledge of natural language. It is necessary to look carefully at both sets of arguments in order to achieve a balanced assessment of the potential and the limitations of LLMs. (shrink)

1. Formal semantics in linguistics -- 2. Generalized quantifier theory -- 3. The interface between syntax and semantics -- 4. Anaphora, discourse, and modality -- 5. Focus, presupposition, and negation -- 6. Tense -- 7. Questions -- 8. Plurals -- 9. Computational semantics -- 10. Lexical semantics -- 11. Semantics and related domains.

The second edition of The Handbook of Contemporary Semantic Theory presents a comprehensive introduction to cutting-edge research in contemporary theoretical and computational semantics. Features completely new content from the first edition of The Handbook of Contemporary Semantic Theory Features contributions by leading semanticists, who introduce core areas of contemporary semantic research, while discussing current research Suitable for graduate students for courses in semantic theory and for advanced researchers as an introduction to current theoretical work.

In this paper, we explore the possibility that machine learning approaches to naturallanguage processing being developed in engineering-oriented computational linguistics may be able to provide specific scientific insights into the nature of human language. We argue that, in principle, machine learning results could inform basic debates about language, in one area at least, and that in practice, existing results may offer initial tentative support for this prospect. Further, results from computational learning theory can inform arguments carried on within linguistic theory (...) as well. (shrink)

We formulate a Curry-typed logic with fine-grained intensionality within Turner’s typed predicate logic. This allows for an elegant presentation of a theory that corresponds to Fox and Lappin’s property theory with curry typing, but without the need for a federation of languages. We then consider how the fine-grained intensionality of this theory can be given an operational interpretation. This interpretation suggests itself as expressions in the theory can be viewed as terms in the untyped lambda-calculus, which provides a model (...) of computation. (shrink)

A BSTRACT. We present Property Theory with Curry Typing, an intensional first-order logic for natural language semantics. PTCT permits fine-grained specifications of meaning. It also supports polymorphic types and separation types.1 We develop an intensional number theory within PTCT in order to represent proportional generalized quantifiers like most. We use the type system and our treatment of generalized quantifiers in natural language to construct a typetheoretic approach to pronominal anaphora that avoids some of the difficulties that undermine previous type-theoretic analyses (...) of this phenomenon. (shrink)

A major challenge for any grammar-driven text understanding system is the resolution of fragments. Basic examples include bare NP answers (1a), where the bare NP John is resolved as the assertion John saw Mary, and sluicing (1b), where the wh-phrase who is interpreted as the question Which student saw John.

Raquel FERN ´ ANDEZ, Jonathan GINZBURG and Shalom LAPPIN Department of Computer Science King’s College London Strand, London WC2R 2LS, UK {raquel,ginzburg,lappin}@dcs.kcl.ac.uk..

I compare several types of knowledge-based and knowledge-poor approaches to anaphora and ellipsis resolution. The former are able to capture fine-grained distinctions that depend on lexical meaning and real world knowledge, but they are generally not robust. The latter show considerable promise for yielding wide coverage systems. However, they consistently miss a small but significant subset of cases that are not accessible to rough-grained techniques of intepretation. I propose a sequenced model which first applies the most computationally efficient and inexpensive (...) methods to resolution and then progresses successively to more costly techniques to deal with cases not handled by previous modules. Confidence measures evaluate the judgements of each component in order to determine which instances of anaphora or ellipsis are to be passed on to the next, more fine-grained subsystem. (shrink)

Much previous work on generation has focused on the general problem of producing lexical strings from abstract semantic representations. We consider generation in the context of a particular task, creating full sentential paraphrases of fragments in dialogue. When the syntactic, semantic and phonological information provided by a dialogue fragment resolution system is made accessible to a generation component, much of the indeterminacy of lexical selection is eliminated.

On the Fregean view of NP's, quantified NP's are represented as operator-variable structures while proper names are constants appearing in argument position. The Generalized Quantifier approach characterizes quantified NP's and names as elements of a unified syntactic category and semantic type. According to the Logicality Thesis, the distinction between quantified NP's, which undergo an operation of quantifier raising to yield operator-variable structures at Logical Form and non-quantified NP's, which appear in situ at LF, corresponds to a difference in logicality status. (...) The former are logical expressions while the latter are not. Using van Benthem's [2, 3] criterion for logicality, I extend the concept of logicality to GQ's. I argue that NP's modified by exception phrases constitute a class of quantified NP's which are heterogeneous with respect to logicality. However, all exception phrase NP's exhibit the syntactic and semantic properties which motivate May to treat quantified NP's as operators at LF. I present a semantic analysis of exception phrases as modifiers of GQ's, and I indicate how this account captures the central semantic properties of exception phrase NP's. I explore the consequences of the logically heterogeneous character of exception phrase NP's for proof theoretic accounts of quantifiers in natural language. The proposed analysis of exception phrase NP's provides support for the GQ approach to the syntax and semantics of NP's. (shrink)

The paper presents Property Theory with Curry Typing (PTCT) where the language of terms and well-formed formulæ are joined by a language of types. In addition to supporting fine-grained intensionality, the basic theory is essentially first-order, so that implementations using the theory can apply standard first-order theorem proving techniques. Some extensions to the type theory are discussed, type polymorphism, and enriching the system with sufficient number theory to account for quantifiers of proportion, such as “most.”.

The application of deep learning methods to problems in natural language processing has generated significant progress across a wide range of natural language processing tasks. For some of these applications, deep learning models now approach or surpass human performance. While the success of this approach has transformed the engineering methods of machine learning in artificial intelligence, the significance of these achievements for the modelling of human learning and representation remains unclear. Deep Learning and Linguistic Representation looks at the application of (...) a variety of deep learning systems to several cognitively interesting NLP tasks. It also considers the extent to which this work illuminates our understanding of the way in which humans acquire and represent linguistic knowledge. Key Features: combines an introduction to deep learning in AI and NLP with current research on Deep Neural Networks in computational linguistics. is self-contained and suitable for teaching in computer science, AI, and cognitive science courses; it does not assume extensive technical training in these areas. provides a compact guide to work on state of the art systems that are producing a revolution across a range of difficult natural language tasks. (shrink)

Sortally incorrect sentences have traditionally been referred to as "category mistakes" (Ryle ()) or "type crossings" (Drange ()). Sortal incorrectness is a ...

Indirect negative evidence is clearly an important way for learners to constrain overgeneralisation, and yet a good learning theoretic analysis has yet to be provided for this, whether in a PAC or a probabilistic identification in the limit framework. In this paper we suggest a theoretical analysis of indirect negative evidence that allows the presence of ungrammatical strings in the input and also accounts for the relationship between grammaticality/acceptability and probability. Given independently justified assumptions about lower bounds on the probabilities (...) of grammatical strings, we establish that a limited number of membership queries of some strings can be probabilistically simulated. (shrink)

In this chapter we consider unsupervised learning from two perspectives. First, we briefly look at its advantages and disadvantages as an engineering technique applied to large corpora in natural language processing. While supervised learning generally achieves greater accuracy with less data, unsupervised learning offers significant savings in the intensive labour required for annotating text. Second, we discuss the possible relevance of unsupervised learning to debates on the cognitive basis of human language acquisition. In this context we explore the implications of (...) recent work on grammar induction for poverty of stimulus arguments that purport to motivate a strong bias model of language learning, commonly formulated as a theory of Universal Grammar (UG). We examine the second issue both as a problem in computational learning theory, and with reference to empirical work on unsupervised Machine Learning (ML) of syntactic structure. We compare two models of learning theory and the place of unsupervised learning within each of them. Looking at recent work on part of speech tagging and the recognition of syntactic structure, we see how far unsupervised ML methods have come in acquiring different kinds of grammatical knowledge from raw text. (shrink)

We present Property Theory with Curry Typing (PTCT), an intensional first-order logic for natural language semantics. PTCT permits fine-grained specifications of meaning. It also supports polymorphic types and separation types. We develop an intensional number theory within PTCT in order to represent proportional generalized quantifiers like “most.” We use the type system and our treatment of generalized quantifiers in natural language to construct a type-theoretic approach to pronominal anaphora that avoids some of the difficulties that undermine previous type-theoretic analyses of (...) this phenomenon. (shrink)

The question of whether it is possible to characterise grammatical knowledge in probabilistic terms is central to determining the relationship of linguistic representation to other cognitive domains. We present a statistical model of grammaticality which maps the probabilities of a statistical model for sentences in parts of the British National Corpus (BNC) into grammaticality scores, using various functions of the parameters of the model. We test this approach with a classifier on test sets containing different levels of syntactic infelicity. With (...) appropriate tuning, the classifiers achieve encouraging levels of accuracy. These experiments suggest that it may be possible to characterise grammaticality judgements in probabilistic terms using an enriched language model. (shrink)

In this paper we address an important issue in the development of an adequate formal theory of underspecified semantics. The tension between expressive power and computational tractability poses an acute problem for any such theory. Generating the full set of resolved scope readings from an underspecified representation produces a combinatorial explosion that undermines the efficiency of these representations. Moreover, Ebert (2005) shows that most current theories of underspecified semantic representation suffer from expressive incompleteness. In previous work we present an account (...) of underspecified scope representations within Property Theory with Curry Typing (PTCT), an intensional first-order theory for natural language semantics. We review this account, and we show that filters applied to the underspecified-scope terms of PTCT permit expressive completeness. While they do not solve the general complexity problem, they do significantly reduce the search space for computing the full set of resolved scope readings in non-worst cases. We explore the role of filters in achieving expressive completeness, and their relationship to the complexity involved in producing full interpretations from underspecified representations. This paper is dedicated to Jim Lambek. (shrink)

Machine learning and statistical methods have yielded impressive results in a wide variety of natural language processing tasks. These advances have generally been regarded as engineering achievements. In fact it is possible to argue that the success of machine learning methods is significant for our understanding of the cognitive basis of language acquisition and processing. Recent work in unsupervised grammar induction is particularly relevant to this issue. It suggests that knowledge of language can be achieved through general learning procedures, and (...) that a richly articulated language faculty is not required to explain its acquisition. (shrink)