Le travail présenté dans cet article s’inscrit dans une recherche qui a pour but la constitution d’un corpus scolaire et le développement d’un outil d’aide à son exploitation à partir de l’annotation de phénomènes linguistiques saillants. Nous nous concentrerons ici sur les écrits produits en fin de classe de CP par des scripteurs encore débutants. L’objet de ce travail est d’explorer les possibilités qu’offre le traitement automatique des langues pour appréhender ces écrits particulièrement éloignés de la norme. L’hypothèse est que (...) la connaissance du contexte de production facilite ce processus. Nous mesurons cet apport au travers d’un exemple de traitement, à savoir le développement d’un outil d’aide à l’annotation de certaines erreurs orthographiques. Après une rapide présentation du projet et des caractéristiques du corpus élaboré, l’article propose un exposé détaillé du module d’annotation de ces erreurs. Il en expose la méthode d’identification et de correction au moyen d’une ressource lexicale de formes phonologiques ainsi que le modèle d’annotation élaboré. (shrink)

Representing provenance information for data is of crucial importance for data reuse. This is in particular the case for language resources such as annotated corpora. NIF has been proposed as an RD...

To maintain the competitive edge and evaluating the needs of the quality app is in the mobile application market. The user’s feedback on these applications plays an essential role in the mobile application development industry. The rapid growth of web technology gave people an opportunity to interact and express their review, rate and share their feedback about applications. In this paper we have scrapped 506259 of user reviews and applications rate from Google Play Store from 14 different categories. The statistical (...) information was measured in the results using different of common machine learning algorithms such as the Logistic Regression, Random Forest Classifier, and Multinomial Naïve Bayes. Different parameters including the accuracy, precision, recall, and F1 score were used to evaluate Bigram, Trigram, and N-gram, and the statistical result of these algorithms was compared. The analysis of each algorithm, one by one, is performed, and the result has been evaluated. It is concluded that logistic regression is the best algorithm for review analysis of the Google Play Store applications. The results have been checked scientifically, and it is found that the accuracy of the logistic regression algorithm for analyzing different reviews based on three classes, i.e., positive, negative, and neutral. (shrink)

In this article, the results of conceptualization of the definition of mind as an object of interdisciplinary applied research are described. The purpose of the theoretical analysis is to generate a methodological discourse suitable for a functional understanding of the mind in the context of the problem of natural language processing as one of the components of developments in the field of artificial intelligence. The conceptual discourse was realized with the help of the author's method of structural-ontological analysis, (...) and developed in the mainstream of the system-methodological tradition of the school of G.P. Shchedrovitsky and intended for descriptive research of subject areas of interdisciplinary objects of scientific study. As a result of the structural and ontological analysis of the super-system, the relevant place and role of the directly studied system (mind) are determined, and its primary process and material are localized and structural and functional connections are described. At the basic level, the mind is conceptualized as an energy process unfolding in a spatio-temporal environment and accompanied by archetypal structuring of neural impulses into images. The genesis of the system is separately analyzed by constructing a structural-ontological matrix that reflects the initial stage of the development of the mind. The primary process is concretized with the help of hetero- and homeostatic dichotomy, and also the most significant features of the consistent transformation of the material of the system and its ascent to verbal morphology are described. The structural-ontological comparison of the functioning of the verbal intelligence with the preverbal level has been carried out. The transformation of neural impulses of needs into words, as verbal units fixing semiotic values is analyzed. Structural-ontological connections that determine the reactive and prospective characteristics of the functioning of the system are disclosed. The position of the chronological primacy of "semiotic readiness" for language with respect to the debut of the latter as an information-sign model of the environment is argued. The hypothesis of domination at the initial stage of the development of the mind of exopsychic functions over endopsychic ones is formulated. The theoretical substantiation of the hypothesis of the inevitability of the ontology of language in the functional understanding of the mind is given, corresponding structural and ontological arguments are given, including those based on the ideas about the information relationship between affects and needs, according to the views of P.V. Simonov. The arguments are presented in favor of the non-alternative methodology of A. Turing in studies of artificial intelligence. (shrink)

Argumentation mining is an application of natural language processing (NLP) that emerged a few years ago and has recently enjoyed considerable popularity, as demonstrated by a series of international workshops and by a rising number of publications at the major conferences and journals of the field. Its goals are to identify argumentation in text or dialogue; to construct representations of the constellation of claims, supporting and attacking moves (in different levels of detail); and to characterize the patterns of (...) reasoning that appear to license the argumentation. Furthermore, recent work also addresses the difficult tasks of evaluating the persuasiveness and quality of arguments. Some of the linguistic genres that are being studied include legal text, student essays, political discourse and debate, newspaper editorials, scientific writing, and others. The book starts with a discussion of the linguistic perspective, characteristics of argumentative language, and their relationship to certain other notions such as subjectivity. Besides the connection to linguistics, argumentation has for a long time been a topic in Artificial Intelligence, where the focus is on devising adequate representations and reasoning formalisms that capture the properties of argumentative exchange. It is generally very difficult to connect the two realms of reasoning and text analysis, but we are convinced that it should be attempted in the long term, and therefore we also touch upon some fundamentals of reasoning approaches. Then the book turns to its focus, the computational side of mining argumentation in text. We first introduce a number of annotated corpora that have been used in the research. From the NLP perspective, argumentation mining shares subtasks with research fields such as subjectivity and sentiment analysis, semantic relation extraction, and discourse parsing. Therefore, many technical approaches are being borrowed from those (and other) fields. We break argumentation mining into a series of subtasks, starting with the preparatory steps of classifying text as argumentative (or not) and segmenting it into elementary units. Then, central steps are the automatic identification of claims, and finding statements that support or oppose the claim. For certain applications, it is also of interest to compute a full structure of an argumentative constellation of statements. Next, we discuss a few steps that try to 'dig deeper': to infer the underlying reasoning pattern for a textual argument, to reconstruct unstated premises (so-called 'enthymemes'), and to evaluate the quality of the argumentation. We also take a brief look at 'the other side' of mining, i.e., the generation or synthesis of argumentative text. The book finishes with a summary of the argumentation mining tasks, a sketch of potential applications, and a—necessarily subjective—outlook for the field. Table of Contents: Preface / Acknowledgments / Introduction / Argumentative Language / Modeling Arguments / Corpus Annotation / Finding Claims / Finding Supporting and Objecting Statements / Deriving the Structure of Argumentation / Assessing Argumentation / Generating Argumentative Text / Summary and Perspectives / Bibliography / Authors' Biographies / Index. (shrink)

One of the fundamental research goals for explanation-based Natural Language Inference (NLI) is to build models that can reason in complex domains through the generation of natural language explanations. However, the methodologies to design and evaluate explanation-based inference models are still poorly informed by theoretical accounts on the nature of explanation. As an attempt to provide an epistemologically grounded characterisation for NLI, this paper focuses on the scientific domain, aiming to bridge the gap between theory and (...) practice on the notion of a scientific explanation. Specifically, the paper combines a detailed survey of the modern accounts of scientific explanation in Philosophy of Science with a systematic analysis of corpora of natural language explanations, clarifying the nature and function of explanatory arguments from both a top-down (categorical) and a bottom-up (corpus-based) perspective. Through a mixture of quantitative and qualitative methodologies, the presented study allows deriving the following main conclusions: (1) Explanations cannot be entirely characterised in terms of inductive or deductive arguments as their main function is to perform unification; (2) An explanation typically cites causes and mechanisms that are responsible for the occurrence of the event to be explained; (3) While natural language explanations possess an intrinsic causal-mechanistic nature, they are not limited to causes and mechanisms, also accounting for pragmatic elements such as definitions, properties and taxonomic relations; (4) Patterns of unification naturally emerge in corpora of explanations even if not intentionally modelled; (5) Unification is realised through a process of abstraction, whose function is to provide the inference mechanism for subsuming the event to be explained under recurring patterns and high-level regularities. The paper contributes to addressing a fundamental gap in classical theoretical accounts on the nature of scientific explanations and their materialisation as linguistic artefacts. This characterisation can support a more principled design and evaluation of explanation-based AI systems which can better interpret, process, and generate natural language explanations. (shrink)

The notion of modularity, introduced by Noam Chomsky and developed with special emphasis on perceptual and linguistic processes by Jerry Fodor in his important book The Modularity of Mind, has provided a significant stimulus to research in cognitive science. This book presents essays in which a diverse group of philosophers, linguists, psycholinguists, and neuroscientists - including both proponents and critics of the modularity hypothesis - address general questions and specific problems related to modularity. Jay L. Garfield is Associate Professor of (...) Philosophy in the School of Communications and Cognitive Science at Hampshire College. (shrink)

The famous advaitic expressions -/- Brahma sat jagat mithya jivo brahma eva na apraha and Asti bhaati priyam namam roopamcheti amsa panchakam AAdya trayam brahma roopam tato dwayam jagat roopam -/- will be analyzed through physics and electronics and interpreted. -/- Four phases of mind, four modes of language acquisition and communication and seven cognitive states of mind participating in human cognitive and language acquisition and communication processes will be identified and discussed. -/- Implications and application of such (...) an identification and analysis to the fields of cognitive sciences, mind-machine modeling, natural language comprehension field of artificial intelligence and physiological psychology will be hinted. A comprehensive modern scientific understanding of human consciousness, mind and mental functions will be presented. (shrink)

The collection of very large text sources has revolutionized the study of natural language, leading to the development of several models of language learning and distributional semantics that extract sophisticated semantic representations of words based on the statistical redundancies contained within natural language. The models treat knowledge as an interaction of processing mechanisms and the structure of language experience. But language experience is often treated agnostically. We report a distributional semantic analysis that shows (...) written language in fiction books varies appreciably between books from the different genres, books from the same genre, and even books written by the same author. Given that current theories assume that word knowledge reflects an interaction between processing mechanisms and the language environment, the analysis shows the need for the field to engage in a more deliberate consideration and curation of the corpora used in computational studies of natural language processing. (shrink)

Knowledge is power. Knowledge about human psychology is increasingly being produced using natural language processing (NLP) and related techniques. The power that accompanies and harnesses this knowledge should be subject to ethical controls and oversight. In this chapter, we address the ethical pitfalls that are likely to be encountered in the context of such research. These pitfalls occur at various stages of the NLP pipeline, including data acquisition, enrichment, analysis, storage, and sharing. We also address secondary uses of (...) the results and tools developed through psychometric NLP, such as profit-driven targeted advertising, political campaigns, and domestic and international psyops. Along the way, we reflect on potential ethical guidelines and considerations that may help researchers navigate these pitfalls. (shrink)

In a recent paper Johan van Benthem reviews earlier work done by himself and colleagues on ‘natural logic’. His paper makes a number of challenging comments on the relationships between traditional logic, modern logic and natural logic. I respond to his challenge, by drawing what I think are the most significant lines dividing traditional logic from modern. The leading difference is in the way logic is expected to be used for checking arguments. For traditionals the checking (...) is local, i.e. separately for each inference step. Between inference steps, several kinds of paraphrasing are allowed. Today we formalise globally: we choose a symbolisation that works for the entire argument, and thus we eliminate intuitive steps and changes of viewpoint during the argument. Frege and Peano recast the logical rules so as to make this possible. I comment also on the traditional assumption that logical processing takes place at the top syntactic level, and I question Johan’s view that natural logic is ‘natural’. (shrink)

Deep Learning has been widely used for tackling challenging natural language processing tasks over the recent years. Similarly, the application of Deep Neural Networks in legal analytics has increased significantly. In this survey, we study the early adaptation of Deep Learning in legal analytics focusing on three main fields; text classification, information extraction, and information retrieval. We focus on the semantic feature representations, a key instrument for the successful application of deep learning in natural language processing. (...) Additionally, we share pre-trained legal word embeddings using the word2vec model over large corpora, comprised legislations from UK, EU, Canada, Australia, USA, and Japan among others. (shrink)

This paper explores two important ways in which the practice of close reading differs from the technique of natural language processing, the use of computer programming to decode, process, and replicate messages within a human language. It does so in order to highlight distinctive features of close reading that are not replicated by natural language processing. The first point of distinction concerns the nature of the meaning generated in each case. While natural language (...) processing proceeds on the principle that a text’s meaning can be deciphered by applying the rules governing the language in which the text is written, close reading is premised on the idea that this meaning lies in the interplay that the text prompts within readers. While the semantic theory of meaning upon which natural language processing programs are based is often taken for granted today, I draw from phenomenological and hermeneutic theories, particularly Wolfgang Iser and Hans-Georg Gadamer, to explain why a different theory of meaning is necessary for understanding the meaning generated by close reading. Second, while natural language processing programs are considered successful when they generate what epistemologists call true beliefs about a text, I argue that close reading aims first and foremost at the development, not of true belief, but of understanding. To develop this distinction, I draw from recent scholarship on the epistemology of education, including work by Duncan Pritchard, to explain how understanding differs from true belief and why attainment of the latter is less educationally significant than the former. (shrink)

In this paper, we present an overview of existing parallel corpora for Automatic Text Simplification (ATS) in different languages focusing on the approach adopted for their construction. We make the main distinction between manual and (semi)–automatic approaches in order to investigate in which respect complex and simple texts vary and whether and how the observed modifications may depend on the underlying approach. To this end, we perform a two-level comparison on Italian corpora, since this is the only language, with (...) the exception of English, for which there are large parallel resources derived through the two approaches considered. The first level of comparison accounts for the main types of sentence transformations occurring in the simplification process, the second one examines the results of a linguistic profiling analysis based on Natural Language Processing techniques and carried out on the original and the simple version of the same texts. For both levels of analysis, we chose to focus our discussion mostly on sentence transformations and linguistic characteristics that pertain to the morpho-syntactic and syntactic structure of the sentence. (shrink)

We describe a knowledge representation and inference formalism, based on an intensional propositional semantic network, in which variables are structures terms consisting of quantifier, type, and other information. This has three important consequences for natural language processing. First, this leads to an extended, more natural formalism whose use and representations are consistent with the use of variables in natural language in two ways: the structure of representations mirrors the structure of the language and allows (...) re-use phenomena such as pronouns and ellipsis. Second, the formalism allows the specification of description subsumption as a partial ordering on related concepts (variable nodes in a semantic network) that relates more general concepts to more specific instances of that concept, as is done in language. Finally, this structured variable representation simplifies the resolution of some representational difficulties with certain classes of natural language sentences, namely, donkey sentences and sentences involving branching quantifiers. The implementation of this formalism is called ANALOG (A NAtural LOGIC) and its utility for natural language processing tasks is illustrated. (shrink)

Spelling can be a source of anxiety for school children and working professionals alike. Yet the spelling of words in English is not as random or chaotic as it is often perceived to be; rather, it is a system based on both meaning and a fascinating linguistic history. Misty Adoniou's public articles on the processes of teaching and learning spelling have garnered an overwhelming response from concerned parents and teachers looking for effective solutions to the problems they face in (...) teaching English spelling to children. Spelling It Out, Revised edition aims to ease anxiety and crush the myth that good spelling comes naturally. Good spelling comes from good teaching. Based on Misty Adoniou's extensive research into spelling learning and instruction, this book encourages children and adults to nurture a curiosity about words, discover their history and, in so doing, understand the logic behind the way they are spelled. (shrink)

An important element of Natural Language Processing is parts of speech tagging. With fine-grained word-class annotations, the word forms in a text can be enhanced and can also be used in downstream processes, such as dependency parsing. The improved search options that tagged data offers also greatly benefit linguists and lexicographers. Natural language processing research is becoming increasingly popular and important as unsupervised learning methods are developed. There are some aspects of the Albanian language that (...) make the creation of a part-of-speech tag set challenging. This research provides a discussion of those issues linguistic phenomena and presents a proposal for a part-of-speech tag set that can adequately represent them. The corpus contains more than 250,000 tokens, each annotated with a medium-sized tag set. The Albanian language’s syntagmatic aspects are adequately represented. Additionally, in this paper are morphologically and part-of-speech tagged corpora for the Albanian language, as well as lemmatize and neural morphological tagger trained on these corpora. Based on the held-out evaluation set, the model achieves 93.65% accuracy on part-of-speech tagging, The morphological tagging rate was 85.31 % and the lemmatization rate was 88.95%. Furthermore, the TF-IDF technique weighs terms and with the scores are highlighted words that have additional information for the Albanian corpus. (shrink)

Most of the existing natural language processing systems for legal texts are developed for the English language. Nevertheless, there are several application domains where multiple versions of the same documents are provided in different languages, especially inside the European Union. One notable example is given by Terms of Service (ToS). In this paper, we compare different approaches to the task of detecting potential unfair clauses in ToS across multiple languages. In particular, after developing an annotated corpus and (...) a machine learning classifier for English, we consider and compare several strategies to extend the system to other languages: building a novel corpus and training a novel machine learning system for each language, from scratch; projecting annotations across documents in different languages, to avoid the creation of novel corpora; translating training documents while keeping the original annotations; translating queries at prediction time and relying on the English system only. An extended experimental evaluation conducted on a large, original dataset indicates that the time-consuming task of re-building a novel annotated corpus for each language can often be avoided with no significant degradation in terms of performance. (shrink)

This paper presents two projects concerned with the application of natural language processing technology for improving communication between Public Administration and citizens. The first project, GIST,is concerned with automatic multilingual generation of instructional texts for form-filling. The second project, TAMIC, aims at providing an interface for interactive access to information, centered on natural language processing and supposed to be used by the clerk but with the active participation of the citizen.

This study investigates the developmental pattern and relationships between oral narrative textual skills, spelling, and written narrative textual skills in monolingual and bilingual language-minority children, L1-Chinese and L2-Italian. The aims were to investigate in monolingual and BLM children: the developmental patterns of oral and writing skills across primary school years; the pattern of relationships between oral narrative textual competence, spelling skills, and written narrative textual competence with age and socio-economic status taken under control. In total, 141 primary (...) class='Hi'>school children from grades 2 to 5 in Central Italy aged between 7 and 11 years obtained scores for oral and written narrative textual competence, spelling accuracy in dictation, and written texts. One-way ANOVA and ANOVA with robust method analyses and Bonferroni’s correction showed that BLM children had poorer spelling skills in dictation and written narrative textual competence than their monolingual peers. After preliminary correlation analysis, the results of hierarchical regression showed that the relationship between oral and written narrative textual competence is completely mediated by spelling accuracy in BLM children. These results suggest that adequate performance in written narrative textual competence depends on adequate spelling accuracy in writing stories. The Sobel test verified the power of this mediation. In monolinguals, the strongest predictor of written narrative textual competence is oral narrative textual competence. This relation is stronger in older children whose spelling skills are automatized. The identified pattern of relationships shows a complex network of oral and written processes. The scarce spelling skills characterizing BLM children may explain why spelling skills determine a low written narrative textual level. Scarce spelling skills absorb cognitive resources, hindering high-level cognitive processes that regulate narrative production. In monolinguals, the medium of writing does not impact narrative textual competence. Children’s oral narrative textual competence easily transfers into their written narrative productions. These findings have implications for the assessment and instruction of literacy skills in young BLM children and their monolingual peers. (shrink)

Exploratory analysis is an area of increasing interest in the computational linguistics arena. Pragmatically speaking, exploratory analysis may be paraphrased as natural language processing by means of analyzing large corpora of text. Concerning the analysis, appropriate means are statistics, on the one hand, and artificial neural networks, on the other hand. As a challenging application area for exploratory analysis of text corpora we may certainly identify text databases, be it information retrieval or information filtering systems. With this paper (...) we present recent findings of exploratory analysis based on both statistical and neural models applied to legal text corpora. Concerning the artificial neural networks, we rely on a model adhering to the unsupervised learning paradigm. This choice appears naturally when taking into account the specific properties of large text corpora where one is faced with the fact that input-output-mappings as required by supervised learning models cannot be provided beforehand to a satisfying extent. This is due to the fact of the highly changing contents of text archives. In a nutshell, artificial neural networks count for their highly robust behavior regarding the parameters for model optimization. In particular, we found statistical classification techniques much more susceptible to minor parameter variations than unsupervised artificial neural networks. In this paper we describe two different lines of research in exploratory analysis. First, we use the classification methods for concept analysis. The general goal is to uncover different meanings of one and the same natural language concept. A task that, obviously, is of specific importance during the creation of thesauri. As a convenient environment to present the results we selected the legal term of neutrality, which is a perfect representative of a concept having a number of highly divergent meanings. Second, we describe the classification methods in the setting of document classification. The ultimate goal in such an application is to uncover semantic similarities of various text documents in order to increase the efficiency of an information retrieval system. In this sense, document classification has its fixed position in information retrieval research from the very beginning. Nowadays renewed massive interest in document classification may be witnessed due to the appearance of large-scale digital libraries. (shrink)

The contract review process can be a costly and time-consuming task for lawyers and clients alike, requiring significant effort to identify and evaluate the legal implications of individual clauses. To address this challenge, we propose the use of natural language processing techniques, specifically text classification based on deontic tags, to streamline the process. Our research question is whether natural language processing techniques, specifically dense vector embeddings, can help semi-automate the contract review process and reduce time and (...) costs for legal professionals reviewing deontic modalities in contracts. In this study, we create a domain-specific dataset and train both baseline and neural network models for contract sentence classification. This approach offers a more efficient and cost-effective solution for contract review, mimicking the work of a lawyer. Our approach achieves an accuracy of 0.90, showcasing its effectiveness in identifying and evaluating individual contract sentences. (shrink)

This Special Issue on Connectionist Models of Human Language Processing provides an opportunity for an appraisal both of specific connectionist models and of the status and utility of connectionist models of language in general. This introduction provides the background for the papers in the Special Issue. The development of connectionist models of language is traced, from their intellectual origins, to the state of current research. Key themes that arise throughout different areas of connectionist psycholinguistics are highlighted, and (...) recent developments in speech processing, morphology, sentence processing, language production, and reading are described. We argue that connectionist psycholinguistics has already had a significant impact on the psychology of language, and that connectionist models are likely to have an important influence on future research. (shrink)

Progress in philosophy is difficult to achieve because our methods are evidentially and rhetorically weak. In the last two decades, experimental philosophers have begun to employ the methods of the social sciences to address philosophical questions. However, the adequacy of these methods has been called into question by repeated failures of replication. Experimental philosophers need to incorporate more robust methods to achieve a multi-modal perspective. In this chapter, we describe and showcase cutting-edge methods for data-mining and visualization. Big data is (...) a useful investigatory tool for moral psychology, and it fits well with the Ramsification method the first author advances in a series of recent papers. The guiding insight of these papers is that we can infer the meaning and structure of concepts from patterns of assertions and inferential associations in natural language. (shrink)

COVID-19 is a disease that affects the quality of life in all aspects. However, the government policy applied in 2020 impacted the lifestyle of the whole world. In this sense, the study of sentiments of people in different countries is a very important task to face future challenges related to lockdown caused by a virus. To contribute to this objective, we have proposed a natural language processing model with the aim to detect positive and negative feelings in open-text (...) answers obtained from a survey in pandemic times. We have proposed a distilBERT transformer model to carry out this task. We have used three approaches to perform a comparison, obtaining for our best model the following average metrics: Accuracy: 0.823, Precision: 0.826, Recall: 0.793 and F1 Score: 0.803. (shrink)

Spelling can be a source of anxiety for school children and working professionals alike. Yet the spelling of words in English is not as random or chaotic as it is often perceived to be; rather, it is a system based on both meaning and a fascinating linguistic history. Misty Adoniou's public articles on the processes of teaching and learning spelling have garnered an overwhelming response from concerned parents and teachers looking for effective solutions to the problems they face in (...) teaching English spelling to children. Spelling It Out aims to ease anxiety and crush the myth that good spelling comes naturally. Good spelling comes from good teaching. Based on Misty Adoniou's extensive research into spelling learning and instruction, this book encourages children and adults to nurture a curiosity about words, discover their history and, in so doing, understand the logic behind the way they are spelled. (shrink)

Natural language processing (NLP) has undergone significant changes in its methods, reflecting advances in computing technology and cognitive research. This article reviews the key stages of the evolution of natural language processing methods. The article touches on the topic of the first NLP systems developed, provides justification for the reasons for the complexity of some processed texts and the possible depth of analysis. In addition, it describes not only NLP methods before and after the GPT revolution, (...) but also current trends and prospects in the field of natural language processing. The article allows us to trace how the idea of natural language text has changed during the development of computer analysis methods, as well as to understand what text is in the mirror of natural language processing, what is really the subject of natural language processing research and what cannot be seen through the eyes of a simple researcher who does not use NLP methods. (shrink)

We introduce a set of biologically and computationally motivated design choices for modeling the learning of language, or of other types of sequential, hierarchically structured experience and behavior, and describe an implemented system that conforms to these choices and is capable of unsupervised learning from raw natural-language corpora. Given a stream of linguistic input, our model incrementally learns a grammar that captures its statistical patterns, which can then be used to parse or generate new data. The grammar (...) constructed in this manner takes the form of a directed weighted graph, whose nodes are recursively (hierarchically) defined patterns over the elements of the input stream. We evaluated the model in seventeen experiments, grouped into five studies, which examined, respectively, (a) the generative ability of grammar learned from a corpus of natural language, (b) the characteristics of the learned representation, (c) sequence segmentation and chunking, (d) artificial grammar learning, and (e) certain types of structure dependence. The model's performance largely vindicates our design choices, suggesting that progress in modeling language acquisition can be made on a broad front—ranging from issues of generativity to the replication of human experimental findings—by bringing biological and computational considerations, as well as lessons from prior efforts, to bear on the modeling approach. (shrink)

Knowing that language is often used as a classifier of human intelligence and the development of systems that understand human language remains a challenge all the time (Kryeziu & Shehu, 2022). Natural Language Processing is a very active field of study, where transformers have a key role. Transformers function based on neural networks and they are increasingly showing promising results. One of the first major contributions to transfer learning in Natural Language Processing was the (...) use of pre-trained word embeddings in 2010 (Joseph, Lev, & Yoshua, 2010). Pre-trained models like ELMo (Matthew, et al., 2018) and BERT (Delvin, et al., 2019) are trained on large corpora of unlabeled text and as a result learning from text representations has achieved good performance on many of the underlying tasks on datasets from different domains. Pre-training in the language model has proven that there has been an improvement in some aspects of natural language processing, based on the paper (Dai & Le, 2015). In present paper, we will pre-train BERT on the task of Masked Language Modeling (MLM) with the Albanian language dataset (alb_dataset) that we have created for this purpose (Kryeziu et al., 2022). We will compare two approaches: training of BERT using the available OSCAR dataset and using our alb_dataset that we have collected. The paper shows some discrepancies during training, especially while evaluating the performance of the model. (shrink)

First published in 1982. Routledge is an imprint of Taylor & Francis, an informa company.

Semantic noise, the effect ensuing from the denotative and thus functional variability exhibited by different terms in different contexts, is a common concern in natural language processing (NLP). While unarguably problematic in specific applications (e.g., certain translation tasks), the main argument of this paper is that failing to observe this linguistic matter of fact as a generative effect rather than as an obstacle, leads to actual obstacles in instances where language model outputs are presented as neutral. Given (...) that a common and long-standing challenge in NLP is the interpretation of ambiguous – i.e., semantically noisy – cases, this article focuses on an exemplar ambiguity-resolution task in NLP: the problem of anaphora in Winograd schemas. The main question considered is: to what extent is the standard approach to disambiguation in NLP subject to a stagnant “image of language”? And, can a transdisciplinary, dynamic approach combining linguistics and philosophy elucidate new perspectives on these possible conceptual shortcomings? In order to answer these questions we explore the term and concept of noise, particularly in its presentation as semantic noise. Owing to its definitional plurality, and sometimes even desirable unspecificity, the term noise is thus used as proof of concept for semantic generativity being an inherent characteristic in linguistic representation, and its concept is used to interrogate assumptions admitted in the resolution of Winograd schemas. The argument is speculative and theoretical in method, and the result is an analysis which provides an account of the fundamentally dialogical and necessarily open-ended effects of semantic noise in natural language. (shrink)

Psycholinguistic experiments show that pronouns tend to be resolved differently depending on whether they occur in main or subordinate clauses. If a pronoun in a subordinate clause has more than one potential antecedent in the main clause, then the pronoun tends to refer to the antecedent which has a certain thematic role (depending on the verb and on the subordinating conjunction). In contrast, pronouns in main clauses tend to refer back to the subject of the previous main clause, and this (...) tendency is not affected by any verbs or conjunctions. In natural language processing, these findings have recently led to a proposal that pronoun resolution systems should have a split architecture, i.e. that they should use different mechanisms for pronoun resolution in the two cases. With the help of two parsed and coreference-annotated corpora, this paper estimates the impact of the split-architecture proposal. The findings of this work are as follows: (1) Subject pronouns in authentic texts behave the same way in main and subordinate clauses. (2) The number of sentences in which a split architecture would behave differently than a system that treats both cases the same way is close to zero. Therefore, a separate treatment of resolution within and across units is unlikely to improve the performance of any system. This result casts a doubt on the split-architecture proposal, and more generally on approaches that directly incorporate psycholinguistic results into performance-oriented algorithms for anaphora resolution without assessing the relative importance of the phenomena that underlie them. (shrink)

Despite its centrality in the philosophy of cognitive science, there has been little prior philosophical work engaging with the notion of representation in contemporary NLP practice. This paper attempts to fill that lacuna: drawing on ideas from cognitive science, I introduce a framework for evaluating the representational claims made about components of neural NLP models, proposing three criteria with which to evaluate whether a component of a model represents a property and operationalising these criteria using probing classifiers, a popular analysis (...) technique in NLP (and deep learning more broadly). The project of operationalising a philosophically-informed notion of representation should be of interest to both philosophers of science and NLP practitioners. It affords philosophers a novel testing-ground for claims about the nature of representation, and helps NLPers organise the large literature on probing experiments, suggesting novel avenues for empirical research. (shrink)

In this chapter we consider unsupervised learning from two perspectives. First, we briefly look at its advantages and disadvantages as an engineering technique applied to large corpora in natural language processing. While supervised learning generally achieves greater accuracy with less data, unsupervised learning offers significant savings in the intensive labour required for annotating text. Second, we discuss the possible relevance of unsupervised learning to debates on the cognitive basis of human language acquisition. In this context we explore (...) the implications of recent work on grammar induction for poverty of stimulus arguments that purport to motivate a strong bias model of language learning, commonly formulated as a theory of Universal Grammar (UG). We examine the second issue both as a problem in computational learning theory, and with reference to empirical work on unsupervised Machine Learning (ML) of syntactic structure. We compare two models of learning theory and the place of unsupervised learning within each of them. Looking at recent work on part of speech tagging and the recognition of syntactic structure, we see how far unsupervised ML methods have come in acquiring different kinds of grammatical knowledge from raw text. (shrink)

SECTION I In 1972, Donald Davison and Gilbert Hannan wrote in the introduction to the volume Semantics of Natural Language: "The success of linguistics in treating natural languages as formal ~yntactic systems has aroused the interest of a number of linguists in a parallel or related development of semantics. For the most part quite independently, many philosophers and logicians have recently been applying formal semantic methods to structures increasingly like natural languages. While differences in training, method (...) and vocabulary tend to veil the fact, philosophers and linguists are converging, it seems, on a common set of interrelated problems. " Davidson and Harman called for an interdisciplinary dialogue of linguists, philosophers and logicians on the semantics of natural language, and during the last ten years such an enterprise has proved extremely fruitful. Thanks to the cooperative effort in these several fields, the last decade has brought about striking progress in our understanding of the semantics of natural language. This work on semantics has typically paid little attention to psychological aspects of meaning. Thus, psychologists or computer scientists working on artificial intelligence were not invited to join the forces in the influential introduction of Semantics of Natural Language. No doubt it was felt that while psychological aspects of language are important in their own right, they are not relevant to our immediate semantic concerns. In the last few years, several linguists and logicians have come to question the fundamental anti-psychological assumptions underlying their theorizing. (shrink)

The global burden of obesity has risen due to various factors, including sociocultural aspects. Social representations (SRs) of obesity could help to understand the problem. Nowadays, social networks activate new social interaction processes and enable the construction of SRs. Tweets can identify mind-sets as cultural reflections of the times. This study aimed to identify widely shared obesity topics on Twitter-Argentina using Natural Language Processing. First, 134,766 Spanish tweets about obesity were collected from August 2021 to July 2022. Next, (...) a geolocation filter removed non-Argentinian messages, leaving 48,149 tweets. The body text was cleaned and prepared for analysis. K-means clustering model was applied and 21 clusters were identified after reaching theoretical saturation. Then, 10 tweets from each cluster were randomly selected to analyse word usage and identify cluster themes. Thus, the main themes were “Gender-related”, “Obesity and family”, “Hate speech and fatphobia”, “Body image, perceptions and feelings”, “Public health actions” and “Obesity as a health-disease process”. These aggregations allowed us to understand obesity as a sociocultural phenomenon, the Argentinian culture-specific discourses surrounding it, and their connections to health. We firmly believe that this valuable information will enlighten the planning of future social-political actions to address this health-disease process. (shrink)

Successful biomedical data mining and information extraction require a complete picture of biological phenomena such as genes, biological processes, and diseases; as these exist on different levels of granularity. To realize this goal, several freely available heterogeneous databases as well as proprietary structured datasets have to be integrated into a single global customizable scheme. We will present a tool to integrate different biological data sources by mapping them to a proprietary biomedical ontology that has been developed for the purposes of (...) making computers understand medical natural language. (shrink)

Developmental systems theory posits that development cannot be segmented by influences acting in isolation, but should be studied through a scientific lens that highlights the complex interactions between these forces over time ( Overton, 2013a ). This poses a unique challenge for developmental psychologists studying complex processes like language development. In this paper, we advocate for the combining of highly sophisticated data collection technologies in an effort to move toward a more systemic approach to studying language development. We (...) investigate the efficiency and appropriateness of combining eye-tracking technology and the LENA (Language Environment Analysis) system, an automated language analysis tool, in an effort to explore the relation between language processing in early development, and external dynamic influences like parent and educator language input in the home and school environments. Eye-tracking allows us to study language processing via eye movement analysis; these eye movements have been linked to both conscious and unconscious cognitive processing, and thus provide one means of evaluating cognitive processes underlying language development that does not require the use of subjective parent reports or checklists. The LENA system, on the other hand, provides automated language output that describes a child’s language-rich environment. In combination, these technologies provide critical information not only about a child’s language processing abilities but also about the complexity of the child’s language environment. Thus, when used in conjunction these technologies allow researchers to explore the nature of interacting systems involved in language development. (shrink)

Human beings express affinity (Shinwa‐kan in Japanese language) in communicating transactive engagements among healthcare providers, patients and healthcare robots. The appearance of healthcare robots and their language capabilities often feature characteristic and appropriate compassionate dialogical functions in human–robot interactions. Elements of healthcare robot configurations comprising its physiognomy and communication properties are founded on the positivist philosophical perspective of being the summation of composite parts, thereby mimicking human persons. This article reviews Mori's theory of the Uncanny Valley and its (...) consequent debates, and examines "Uncanny" relations with generating healthcare robot conversational content with artificial affective communication (AAC) using natural language processing. With healthcare robots provoking influential physical composition and sensory expressions, the relations in human–healthcare robot transactive engagements are argued as supportive of the design and development in natural language processing. This implies that maintaining human–healthcare robot interaction and assessing the eeriness situations explained in the Uncanny Valley theory are crucial positions for healthcare robot functioning as a valuable commodity in health care. As such, physical features, language capabilities and mobility of healthcare robots establish the primacy of the AAC with natural language processing as integral to healthcare robot–human healthcare practice. (shrink)

Ideas about human consciousness and mental functions will be analyzed and developed using cognitive science information available in the Upanishads, Brahmajnaana, Advaita and Dvaita schools of thought. -/- The analysis and development so done will be used to theorize and give scheme of human language acquisition and communication process clubbing with Sabdabrahma Siddhanta/Sphota Vaada which put forward infrasonic wave oscillator issuing pulses in infrasonic range and are reflected as brain waves. -/- Thus a brain-wave modulation/demodulation model of human (...) class='Hi'>language acquisition and communication will be advanced and put forward. Application of this study in mind-machine modeling, natural language comprehension field of artificial intelligence will also be hinted. -/- . (shrink)

Spelling errors are ubiquitous in all writing systems. Most studies exploring spelling errors focused on the phonological plausibility of errors. However, unlike typical pseudohomophones, spelling errors occur in naturally produced written language. We investigated the time course of recognition of the most frequent orthographic errors in Russian and the effect of word frequency on this process. During event-related potentials recording, 26 native Russian speakers silently read high-frequency correctly spelled words, low-frequency correctly spelled words, high-frequency words with errors, and low-frequency (...) words with errors. The amplitude of P200 was more positive for correctly spelled words than for misspelled words and did not depend on the frequency of the words. In addition, in the 350–500-ms time window, we found a more negative response for misspelled words than for correctly spelled words in parietal–temporal-occipital regions regardless of word frequency. Considering our results in the context of a dual-route model, we concluded that recognizing misspelled high-frequency and low-frequency words involves common orthographic and phonological processes associated with P200 and N400 components such as whole word orthography processing and activation of phonological representations correspondingly. However, at the 500–700 ms stage, error recognition depends on the word frequency. One possible explanation for these differences could be that at the 500–700 ms stage recognition of high-frequency misspelled and correctly spelled words shifts from phonological to orthographic processes, while low-frequency misspelled words are accompanied by more prolonged phonological activation. We believe these processes may be associated with different ERP components P300 and N400, reflecting a temporal overlap between categorization processes based on orthographic properties for high-frequency words and phonological processes for low-frequency words. Therefore, our results complement existing reading models and demonstrate that the neuronal underpinnings of spelling error recognition during reading may depend on word frequency. (shrink)

The central hypothesis of the collaboration between Language and Computing (L&C) and the Institute for Formal Ontology and Medical Information Science (IFOMIS) is that the methodology and conceptual rigor of a philosophically inspired formal ontology greatly benefits application ontologies.[1] To this end LinKBase®, L&C’s ontology, which is designed to integrate and reason across various external databases simultaneously, has been submitted to the conceptual demands of IFOMIS’s Basic Formal Ontology (BFO).[2] With this project we aim to move beyond the level (...) of controlled vocabularies to yield an ontology with the ability to support reasoning applications. Our general procedure has been the implementation of a meta-ontological definition space in which the definitions of all the concepts and relations in LinKBase® are standardized in a framework of first-order logic. In this paper we describe how this standardization has already led to an improvement in the LinKBase® structure that allows for a greater degree of internal coherence than ever before possible. We then show the use of this philosophical standardization for the purpose of mapping external databases to one another, using LinKBase® as translation hub, with a greater degree of success than possible hitherto. We demonstrate how this offers a genuine advance over other application ontologies that have not submitted themselves to the demands of philosophical scrutiny. (shrink)

Traditional methods for deriving property-based representations of concepts from text have focused on either extracting only a subset of possible relation types, such as hyponymy/hypernymy (e.g., car is-a vehicle) or meronymy/metonymy (e.g., car has wheels), or unspecified relations (e.g., car—petrol). We propose a system for the challenging task of automatic, large-scale acquisition of unconstrained, human-like property norms from large text corpora, and discuss the theoretical implications of such a system. We employ syntactic, semantic, and encyclopedic information to guide our extraction, (...) yielding concept-relation-feature triples (e.g., car be fast, car require petrol, car cause pollution), which approximate property-based conceptual representations. Our novel method extracts candidate triples from parsed corpora (Wikipedia and the British National Corpus) using syntactically and grammatically motivated rules, then reweights triples with a linear combination of their frequency and four statistical metrics. We assess our system output in three ways: lexical comparison with norms derived from human-generated property norm data, direct evaluation by four human judges, and a semantic distance comparison with both WordNet similarity data and human-judged concept similarity ratings. Our system offers a viable and performant method of plausible triple extraction: Our lexical comparison shows comparable performance to the current state-of-the-art, while subsequent evaluations exhibit the human-like character of our generated properties. (shrink)