While rules and exemplars are usually viewed as opposites, this paper argues that they form end points of the same distribution. By representing both rules and exemplars as (partial) trees, we can take into account the fluid middle ground between the two extremes. This insight is the starting point for a new theory of language learning that is based on the following idea: If a language learner does not know which phrase‐structure trees should be assigned to initial sentences, s/he allows (...) (implicitly) for all possible trees and lets linguistic experience decide which is the “best” tree for each sentence. The best tree is obtained by maximizing “structural analogy” between a sentence and previous sentences, which is formalized by the most probable shortest combination of subtrees from all trees of previous sentences. Corpus‐based experiments with this model on the Penn Treebank and the Childes database indicate that it can learn both exemplar‐based and rule‐based aspects of language, ranging from phrasal verbs to auxiliary fronting. By having learned the syntactic structures of sentences, we have also learned the grammar implicit in these structures, which can in turn be used to produce new sentences. We show that our model mimicks children’s language development from item‐based constructions to abstract constructions, and that the model can simulate some of the errors made by children in producing complex questions. (shrink)

Modeling law search and retrieval as prediction problems has recently emerged as a predominant approach in law intelligence. Focusing on the law article retrieval task, we present a deep learning framework named LamBERTa, which is designed for civil-law codes, and specifically trained on the Italian civil code. To our knowledge, this is the first study proposing an advanced approach to law article prediction for the Italian legal system based on a BERT (Bidirectional Encoder Representations from Transformers) learning framework, which has (...) recently attracted increased attention among deep learning approaches, showing outstanding effectiveness in several natural language processing and learning tasks. We define LamBERTa models by fine-tuning an Italian pre-trained BERT on the Italian civil code or its portions, for law article retrieval as a classification task. One key aspect of our LamBERTa framework is that we conceived it to address an extreme classification scenario, which is characterized by a high number of classes, the few-shot learning problem, and the lack of test query benchmarks for Italian legal prediction tasks. To solve such issues, we define different methods for the unsupervised labeling of the law articles, which can in principle be applied to any law article code system. We provide insights into the explainability and interpretability of our LamBERTa models, and we present an extensive experimental analysis over query sets of different type, for single-label as well as multi-label evaluation tasks. Empirical evidence has shown the effectiveness of LamBERTa, and also its superiority against widely used deep-learning text classifiers and a few-shot learner conceived for an attribute-aware prediction task. (shrink)

Artificial Intelligence in the form of different machine learning models is applied to Big Data as a way to turn data into valuable knowledge. The rhetoric is that ensuing predictions work well—with a high degree of autonomy and automation. We argue that we need to analyze the process of applying machine learning in depth and highlight at what point human knowledge production takes place in seemingly autonomous work. This article reintroduces classification theory as an important framework for understanding such seemingly (...) invisible knowledge production in the machine learning development and design processes. We suggest a framework for studying such classification closely tied to different steps in the work process and exemplify the framework on two experiments with machine learning applied to Facebook data from one of our labs. By doing so we demonstrate ways in which classification and potential discrimination take place in even seemingly unsupervised and autonomous models. Moving away from concepts of non-supervision and autonomy enable us to understand the underlying classificatory dispositifs in the work process and that this form of analysis constitutes a first step towards governance of artificial intelligence. (shrink)

Contemporary Artificial Intelligence methods, especially their subset-deep learning, are finding their way to successful implementations in the detection and classification of intrusions at the network level. This paper presents an intrusion detection mechanism that leverages Deep AutoEncoder and several Deep Decoders for unsupervised classification. This work incorporates multiple network topology setups for comparative studies. The efficiency of the proposed topologies is validated on two established benchmark datasets: UNSW-NB15 and NetML-2020. The results of their analysis are discussed in terms of classification (...) accuracy, detection rate, false-positive rate, negative predictive value, Matthews correlation coefficient and F1-score. Furthermore, comparing against the state-of-the-art methods used for network intrusion detection is also disclosed. (shrink)

In this chapter we consider unsupervised learning from two perspectives. First, we briefly look at its advantages and disadvantages as an engineering technique applied to large corpora in natural language processing. While supervised learning generally achieves greater accuracy with less data, unsupervised learning offers significant savings in the intensive labour required for annotating text. Second, we discuss the possible relevance of unsupervised learning to debates on the cognitive basis of human language acquisition. In this context we explore the implications of (...) recent work on grammar induction for poverty of stimulus arguments that purport to motivate a strong bias model of language learning, commonly formulated as a theory of Universal Grammar (UG). We examine the second issue both as a problem in computational learning theory, and with reference to empirical work on unsupervised Machine Learning (ML) of syntactic structure. We compare two models of learning theory and the place of unsupervised learning within each of them. Looking at recent work on part of speech tagging and the recognition of syntactic structure, we see how far unsupervised ML methods have come in acquiring different kinds of grammatical knowledge from raw text. (shrink)

We describe a way of modeling high‐dimensional data vectors by using an unsupervised, nonlinear, multilayer neural network in which the activity of each neuron‐like unit makes an additive contribution to a global energy score that indicates how surprised the network is by the data vector. The connection weights that determine how the activity of each unit depends on the activities in earlier layers are learned by minimizing the energy assigned to data vectors that are actually observed and maximizing the energy (...) assigned to “confabulations” that are generated by perturbing an observed data vector in a direction that decreases its energy under the current model. (shrink)

The automated identification of national implementations of European directives by text similarity techniques has shown promising preliminary results. Previous works have proposed and utilized unsupervised lexical and semantic similarity techniques based on vector space models, latent semantic analysis and topic models. However, these techniques were evaluated on a small multilingual corpus of directives and NIMs. In this paper, we utilize word and paragraph embedding models learned by shallow neural networks from a multilingual legal corpus of European directives and national legislation (...) to develop unsupervised semantic similarity systems to identify transpositions. We evaluate these models and compare their results with the previous unsupervised methods on a multilingual test corpus of 43 Directives and their corresponding NIMs. We also develop supervised machine learning models to identify transpositions and compare their performance with different feature sets. (shrink)

Scientists increasingly approach the world through machine learning techniques, but philosophers of science often question their epistemic status. Some philosophers have argued that the use of unsupervised clustering algorithms is more justified than the use of supervised classification, because supervised classification is more biased, and because (parametric) simplicity plays a different and more interesting role in unsupervised clustering. I call these arguments the No-Bias Argument and the Simplicity-Truth Argument. I show how both arguments are fallacious and how, on the contrary, (...) the use of supervised classification is at least as justified as the use of unsupervised clustering. (shrink)

We describe a linguistic pattern acquisition algorithm that learns, in an unsupervised fashion, a streamlined representation of corpus data. This is achieved by compactly coding recursively structured constituent patterns, and by placing strings that have an identical backbone and similar context structure into the same equivalence class. The resulting representations constitute an efficient encoding of linguistic knowledge and support systematic generalization to unseen sentences.

To learn a visual code in an unsupervised manner, one may attempt to capture those features of the stimulus set that would contribute significantly to a statistically efficient representation. Paradoxically, all the candidate features in this approach need to be known before statistics over them can be computed. This paradox may be circumvented by confining the repertoire of candidate features to actual scene fragments, which resemble the “what+where” receptive fields found in the ventral visual stream in primates. We describe a (...) single-layer network that learns such fragments from unsegmented raw images of structured objects. The learning method combines fast imprinting in the feedforward stream with lateral interactions to achieve single-epoch unsupervised acquisition of spatially localized features that can support systematic treatment of structured objects [1]. (shrink)

Unsupervised learning algorithms are widely used for many important statistical tasks with numerous applications in science and industry. Yet despite their prevalence, they have attracted remarkably little philosophical scrutiny to date. This stands in stark contrast to supervised and reinforcement learning algorithms, which have been widely studied and critically evaluated, often with an emphasis on ethical concerns. In this article, I analyze three canonical unsupervised learning problems: clustering, abstraction, and generative modeling. I argue that these methods raise unique epistemological and (...) ontological questions, providing data-driven tools for discovering natural kinds and distinguishing essence from contingency. This analysis goes some way toward filling the lacuna in contemporary philosophical discourse on unsupervised learning, as well as bringing conceptual unity to a heterogeneous field more often described by what it isnot(i.e., supervised or reinforcement learning) than by what itis. I submit that unsupervised learning is not just a legitimate subject of philosophical inquiry but perhaps the most fundamental branch of all AI. However, an uncritical overreliance on unsupervised methods poses major epistemic and ethical risks. I conclude by advocating for a pragmatic, error-statistical approach that embraces the opportunities and mitigates the challenges posed by this powerful class of algorithms. (shrink)

Unsupervised statistical learning is the standard setting for the development of the only advanced visual system that is both highly sophisticated and versatile, and extensively studied: that of monkeys and humans. In this extended abstract, we invoke philosophical observations, computational arguments, behavioral data and neurobiological findings to explain why computer vision researchers should care about (1) unsupervised learning, (2) statistical inference, and (3) the visual brain. We then outline a neuromorphic approach to structural primitive learning motivated by these considerations, survey (...) a range of neurobiological findings and behavioral data consistent with it, and conclude by mentioning some of the more challenging directions for future research. (shrink)

We address the problem of predicting how people will spontaneously divide into groups a set of novel items. This is a process akin to perceptual organization. We therefore employ the simplicity principle from perceptual organization to propose a simplicity model of unconstrained spontaneous grouping. The simplicity model predicts that people would prefer the categories for a set of novel items that provide the simplest encoding of these items. Classification predictions are derived from the model without information either about the number (...) of categories sought or information about the distributional properties of the objects to be classified. These features of the simplicity model distinguish it from other models in unsupervised categorization (where, for example, the number of categories sought is determined via a free parameter), and we discuss how these computational differences are related to differences in modeling objectives. The predictions of the simplicity model are validated in four experiments. We also discuss the significance of simplicity in cognitive modeling more generally. (shrink)

We describe a pattern acquisition algorithm that learns, in an unsupervised fashion, a streamlined representation of linguistic structures from a plain natural-language corpus. This paper addresses the issues of learning structured knowledge from a large-scale natural language data set, and of generalization to unseen text. The implemented algorithm represents sentences as paths on a graph whose vertices are words. Significant patterns, determined by recursive context-sensitive statistical inference, form new vertices. Linguistic constructions are represented by trees composed of significant patterns and (...) their associated equivalence classes. An input module allows the algorithm to be subjected to a standard test of English as a Second Language profi- ciency. The results are encouraging: the model attains a level of performance considered to be “intermediate” for 9th-grade students, despite having been trained on a corpus containing transcribed speech of parents directed to small children. (shrink)

In the domain of legal information retrieval, an important challenge is to compute similarity between two legal documents. Precedents play an important role in The Common Law system, where lawyers need to frequently refer to relevant prior cases. Measuring document similarity is one of the most crucial aspects of any document retrieval system which decides the speed, scalability and accuracy of the system. Text-based and network-based methods for computing similarity among case reports have already been proposed in prior works but (...) not without a few pitfalls. Since legal citation networks are generally highly disconnected, network based metrics are not suited for them. Till date, only a few text-based and predominant embedding based methods have been employed, for instance, TF-IDF based approaches, Word2Vec and Doc2Vec based approaches. We investigate the performance of 56 different methodologies for computing textual similarity across court case statements when applied on a dataset of Indian Supreme Court Cases. Among the 56 different methods, thirty are adaptations of existing methods and twenty-six are our proposed methods. The methods studied include models such as BERT and Law2Vec. It is observed that the more traditional methods that rely on a bag-of-words representation performs better than the more advanced context-aware methods for computing document-level similarity. Finally we nominate, via empirical validation, five of our best performing methods as appropriate for measuring similarity between case reports. Among these five, two are adaptations of existing methods and the other three are our proposed methods. (shrink)

Collaborative learning has recently achieved very significant results. It still suffers, however, from several issues, including the type of information that needs to be exchanged, the criteria for stopping and how to choose the right collaborators. We aim in this paper to improve the quality of the collaboration and to resolve these issues via a novel approach inspired by Optimal Transport theory. More specifically, the objective function for the exchange of information is based on the Wasserstein distance, with a bidirectional (...) transport of information between collaborators. This formulation allows to learns a stopping criterion and provide a criterion to choose the best collaborators. Extensive experiments are conducted on multiple data-sets to evaluate the proposed approach. (shrink)

The ability to anticipate team members' actions enables joint action towards a common goal. Task knowledge and mental simulation allow for anticipating other agents' actions and for making inferences about their underlying mental representations. In human–AI teams, providing AI agents with anticipatory mechanisms can facilitate collaboration and successful execution of joint action. This paper presents a computational cognitive model demonstrating mental simulation of operators' mental models of a situation and anticipation of their behavior. The work proposes two successive steps: (1) (...) A hierarchical cluster algorithm is applied to recognize patterns of behavior among pilots. These behavioral clusters are used to derive commonalities in situation models from empirical data (N = 13 pilots). (2) An ACT-R (adaptive control of thought - rational) cognitive model is implemented to mentally simulate different possible outcomes of action decisions and timing of a pilot. model tracing of ACT-R allows following up on operators' individual actions. Two models are implemented using the symbolic representations of ACT-R: one simulating normative behavior and the other by simulating individual differences and using subsymbolic learning. Model performance is analyzed by a comparison of both models. Results indicate the improved performance of the individual differences over the normative model and are discussed regarding implications for cognitive assistance capable of anticipating operator behavior. (shrink)

We outline an unsupervised language acquisition algorithm and offer some psycholinguistic support for a model based on it. Our approach resembles the Construction Grammar in its general philosophy, and the Tree Adjoining Grammar in its computational characteristics. The model is trained on a corpus of transcribed child-directed speech (CHILDES). The model’s ability to process novel inputs makes it capable of taking various standard tests of English that rely on forced-choice judgment and on magnitude estimation of linguistic acceptability. We report encouraging (...) results from several such tests, and discuss the limitations revealed by other tests in our present method of dealing with novel stimuli. (shrink)

tic structure. Determining the semantic roles of a verb’s dependents is an important step in natural..

This study investigates the types of misinformation spread on Twitter that evokes scientific authority or evidence when making false claims about the antimalarial drug hydroxychloroquine as a treatment for COVID-19. Specifically, we examined tweets generated after former U.S. President Donald Trump retweeted misinformation about the drug using an unsupervised machine learning approach called the biterm topic model that is used to cluster tweets into misinformation topics based on textual similarity. The top 10 tweets from each topic cluster were content coded (...) for three types of misinformation categories related to scientific authority: medical endorsements of hydroxychloroquine, scientific information used to support hydroxychloroquine’s use, and a comparison group that included scientific evidence opposing hydroxychloroquine’s use. Results show a much higher volume of tweets featuring medical endorsements and use of supportive scientific information compared to accurate and updated scientific evidence, that misinformation-related tweets propagated for a longer time frame, and the majority of hydroxychloroquine Twitter discourse expressed positive views about the drug. Metadata from Twitter accounts found that prominent users within misinformation discourse were more likely to have media or political affiliation and explicitly expressed support for President Trump. Conversely, prominent accounts within the scientific opposition discourse primarily consisted of medical doctors or scientists but had far less influence in the Twitter discourse. Implications of these findings and connections to related social media research are discussed, as well as cognitive mechanisms for understanding susceptibility to misinformation and strategies to combat misinformation spread via online platforms. (shrink)

We compare our model of unsupervised learning of linguistic structures, ADIOS [1], to some recent work in computational linguistics and in grammar theory. Our approach resembles the Construction Grammar in its general philosophy (e.g., in its reliance on structural generalizations rather than on syntax projected by the lexicon, as in the current generative theories), and the Tree Adjoining Grammar in its computational characteristics (e.g., in its apparent affinity with Mildly Context Sensitive Languages). The representations learned by our algorithm are truly (...) emergent from the (unannotated) corpus data, whereas those found in published works on cognitive and construction grammars and on TAGs are hand-tailored. Thus, our results complement and extend both the computational and the more linguistically oriented research into language acquisition. We conclude by suggesting how empirical and formal study of language can be best integrated. (shrink)

Extreme Learning Machine is widely used in various fields because of its fast training and high accuracy. However, it does not primarily work well for Domain Adaptation in which there are many annotated data from auxiliary domain and few even no annotated data in target domain. In this paper, we propose a new variant of ELM called Discriminative Extreme Learning Machine with Cross-Domain Mean Approximation for unsupervised domain adaptation. It introduces Cross-Domain Mean Approximation into the hidden layer of ELM to (...) reduce distribution discrepancy between domains for domain bias elimination, which is conducive to train a high accuracy ELM on annotated data from auxiliary domains for target tasks. Linear Discriminative Analysis is also adopted to improve the discrimination of learned model and obtain higher accuracy. Moreover, we further provide a Discriminative Kernel Extreme Learning Machine with Cross-Domain Mean Approximation as the kernelization extension of DELM-CDMA. Some experiments are performed to investigate the proposed approach, and the result shows that DELM-CDMA and DKELM-CDMA could effectively extend ELM suitable for domain adaptation and outperform ELM and many other domain adaptation approaches. (shrink)

As it is well known, mobile phones have become a basic gadget for any individual that usually stores sensitive information. This mainly motivates the increase in the number of attacks aimed at jeopardizing smartphones, being an extreme concern above all on Android OS, which is the most popular platform in the market. Consequently, a strong effort has been devoted for mitigating mentioned incidents in recent years, even though few researchers have addressed the application of visualization techniques for the analysis of (...) malware. Within this field, the present work proposes the extension of a new technique called Hybrid Unsupervised Exploratory Plots to visualize Android malware datasets. More precisely, the novel Beta-Hebbian Learning (BHL) method is applied for the first time and validated under the frame of Hybrid Unsupervised Exploratory Plots, in conjunction with clustering methods. The informative visualization achieved provides a picture of the structure of the malware families, allowing subsequent analysis of their organization. To validate the Hybrid Unsupervised Exploratory Plot extension and its tuning, the popular Android Malware Genome dataset has been used in the experimental setting. Promising results have been obtained, suggesting that BHL applied in combination with clustering techniques in Hybrid Unsupervised Exploratory Plots are a viable resource for the visualization of malware families. (shrink)

The COVID-19 pandemic was a global health crisis that required US residents to understand the phenomenon, interpret the cues, and make sense within their environment. Therefore, how the communication of COVID-19 was framed to stakeholders during the early stages of the pandemic became important to guide them through specific actions in their state and subsequently with the sensemaking process. The present study examines which frames were emphasized in the states' press releases on policies and other COVID information to influence stakeholders (...) on what to focus on to help with sensemaking during the crisis. We conducted content analysis on 602 press releases from 50 US states using an unsupervised machine learning approach called Latent Dirichlet Allocation (LDA). The results show that health communication using press releases to help the public make sense of the crisis were framed to include health frames as well as economic frames. Health communication messages are typically framed with health and safety measures; however, this study shows that economic frames were emphasized more than public health frames in the government's health communication for COVID-19, which forced both large and small businesses to engage in specific socially responsible activities that were previously voluntary to support public health safety. (shrink)

We point out that a certain complex compact manifold constructed by Lieberman has the dimensional order property, and has U-rank different from Morley rank. We also give a sufficient condition for a Kahler manifold to be totally degenerate (that is, to be an indiscernible set, in its canonical language) and point out that there are K3 surfaces which satisfy these conditions.