This paper addresses the treatment of noise caused by word omissions in a corpus of school writings, in order to facilitate their subsequent automatic processing. While a normalization step may facilitate the processing of these texts, certain linguistic expressions remain challenging to comprehend, particularly in instances where the writer omits words from the text. The present contribution proposes three automatic and semi-automatic potential solutions to this problem. The first method employs a "mask" token in the form of xxx. The second (...) is a semi-automatic approach whereby each morpho-syntactic category proposed during normalization is replaced by the corresponding "prototypical word." The third involves a FlauBERT method, using this language model to "reconstruct" the most probable token in the text. The three methods are evaluated quantitatively, and the results obtained using method 3, which proved to be the most effective in the context of our research, are also presented qualitatively. (shrink)

Although Dagur and Alsatian represent two typologically distant language families, they share several similarities: both languages are endangered, do not have a unified spelling system, and have few available digital corpora. Given these challenges, the main aim of this article is to compare the noise in corpora for these languages and its impact on part-of-speech (POS) annotation and tagging. We first discuss what strategies can be used to reduce the noise due to spelling inconsistencies observed during corpus collection, using Dagur (...) as an example. We then observe that the distributions of POS trigrams in the manually annotated Dagur and Alsatian corpora are similar to those of typologically related languages in UD v2.12, laying the foundations for experimenting with zero-shot transfer approaches for automatic POS tagging. We evaluate some simple noise reduction strategies for POS tagging using the example of the Alsatian dialects and relying on their proximity to standard German. The results obtained confirm the important role of linguistic proximity in automatic POS tagging and the effectiveness of the proposed data transformation method. However, they also invite further interpretation of the capacities of multilingual models. (shrink)

Speech-to-Text applications have made dazzling progress in recent years (e.g. Whisper). However, since they are usually intended to generate texts conform to written standards, they tend to blur marks of an oral nature (e.g. repetitions, pauses in the stream of words, phatics like er…). Thus, even if such applications suggest huge benefits in terms of working time as well as transcription accuracy, they remain inadequate for verbal exchanges analysis. Relying on a sample of transcripts from medical consultations anticipating an awake (...) brain-surgery (a research now in progress), this study aims at three goals: (i) drawing up an inventory of difficulties revealed by the use of Whisper from a speech analyst perspective; (ii) clarifying the origins of these difficulties; (iii) suggesting ways to overcome them. (shrink)

There are a number of tools for extracting textual content on the Internet. Many of these tools were designed by researchers in the field of natural language processing, and are available as “packages”, which are sets of code files that developers have no trouble using but that may prove out of reach for laymen. Independent software solutions are few, and unlikely to satisfy the needs of researchers in the field of discourse analysis. In this article, we introduce a semi-automated, textual (...) data extraction software, Le Détricoteur, which aims to meet the epistemological requirements of discourse analysis while also being accessible to non-specialist users who are not familiar with coding. (shrink)

Corpora in the didactics of grammar are little studied, even though they appear to be central in supporting the teaching and, above all, the learning of grammatical notions. After clarifying their role in teaching and identifying the characteristics of a well-constructed corpus, we analyze the corpus of the popular digital game PP L’archer from the Alloprof platform. We find that it contains a limited number of sentences and that they lack diversity in their construction. We discuss the relevance of our (...) findings for improving the online version of the game and, more broadly, for improving the grammatical corpora presented to students. (shrink)

Contexte Ce volume naît de la richesse des discussions et des échanges qui ont eu lieu lors de la journée d’étude intitulée « Bruit de fond ou valeur ajoutée? Gérer le bruit lors des traitements informatiques des corpus linguistiques », qui s’est tenue à Grenoble le 28 avril 2023. Cet événement, organisé conjointement par l’Université Grenoble Alpes et l’Université Sapienza de Rome, a vu la collaboration des laboratoires LIDILEM (Univ. Grenoble Alpes) et ECP (Univ. Lumière Lyon 2) ainsi que...

The digitization of ancient texts and tools for processing written corpora now make it possible to refine linguistic research in long diachrony. The aim of this analysis is to use the Ngram Viewer to study changes in the frequency of blue and its shades (studied through various terms, names of colors, pigments and dyes, e.g. bleu céleste, bleu roi, bleu de Paris, safre) in the French sub-collection of the Google Books corpus, which includes works published between 1500 and 2022. As (...) this study follows on from our previous quantitative research (Kaliska & Gostkowska 2022), we will be able to draw parallels between the results obtained from the corpus and the data obtained from the dictionaries, on the one hand, and to compare them with historical findings, which tell us that blue, so popular today, was once marginalized. (shrink)

This article investigates the role of Scholarly Digital Editions in advancing textual scholarship, with a focus on Judeo-Arabic, a unique linguistic variety using Hebrew script to represent Arabic. It proposes a stratified model for Scholarly Digital Editions to handle Judeo-Arabic’s orthographic complexities, including transcription, script conversion, normalization, and metadata integration. The study highlights the critical role of managing noise in textual editing, particularly during data acquisition, where any distortion or omission of orthographic elements can compromise scholarly integrity. A case study (...) of a medieval Judeo-Arabic manuscript illustrates the model’s application, highlighting how it preserves textual integrity and enriches scholarly analysis. This methodology not only addresses Judeo-Arabic specific challenges but is also adaptable to other allographic systems. (shrink)

In the field of NLP, “noise” is a notion with many different meanings. This is also true in fields of linguistics in which the analysis of ecological data is central.In studies involving corpus linguistic methods, noise management is an essential process for data collection, data structuration and data analysis. Paradoxically, this step is almost never developed, or completely ignored.In this paper, we propose to focus on the management of noise during the various stages classically defined in the processing of an (...) oral corpus: data structuration (including transcription), data enrichment (including annotation) and data analysis.We show that noise management is a cross-cutting issue: it needs to be taken into account during the phases of structuration, data enrichment and analysis of linguistic corpora. More specifically, the creation of corpora of children’s productions leads us to manage a particular type of noise: the variation regarding the so-called “standard” forms. For noise management, we show that there is no ideal methodology: noise management is specific to each project. Moreover, decisions taken at one stage necessarily influence the others: a noise generated during data structuration may cause a silence during automatic processing; a noise “cleaned up” during the processing phase may generate a silence during analysis. (shrink)

This article explores the notion of noise in emotion annotation, within the context of measuring the acceptability of innovations. For the task under study, using annotator disagreement as a measure of noise tends to overestimate its presence in the data. An alternative approach is proposed, which involves modeling the annotation process through a probabilistic model. This method accounts for the variability in annotations, though it does limit the ability to accurately assess the noise component.

Dans le contexte d’une révision de l’élargissement de l’UE aux Balkans, la couverture médiatique du Monténégro, l’un des principaux acteurs de ce processus, dans la presse allemande est devenue plus présente ces dernières années. Cette recherche a pour but d’explorer les représentations du Monténégro dans quatre journaux allemands de premier plan. La contribution propose une approche d’analyse de données textuelles appliquée à un corpus composé d’articles consacrés au Monténégro entre 2016 et 2023. Grâce à des méthodes statistiques de texte telles (...) que le z-score et les calculs de cooccurrences, l’article explore l’interaction complexe des facteurs sociopolitiques, historiques et internationaux qui influencent l’image du pays dans la presse allemande. Les résultats révèlent des variations significatives dans les perspectives et les niveaux de représentation entre les journaux, ce qui est révélateur de leur position idéologique. Malgré sa taille géographique relativement modeste, le Monténégro suscite un niveau d’attention modéré dans ces journaux allemands, comme le montrent les marqueurs linguistiques identifiés par l’analyse statistique. En tant qu’étude pionnière, cet article contribue à une meilleure compréhension de la manière dont l’image et la perception de pays comme le Monténégro sont façonnées dans les contextes de la presse étrangère. (shrink)

This paper presents the results of a research work that aims to determine whether the upstream OCR correction can significantly improve the results of the Named Entity Recognition (NER) task. The experiments were applied to the ELTeC and Very Big Library (TGB) corpora. Our objective is to establish (i) a typology of OCR contaminations from the Kraken and Tesseract tools and (ii) a typology of errors in the automatic correction produced by the JamSpell tool. As part of our evaluation, we (...) study the intersections between the NEs detected by the spaCy tool in the reference texts and the two OCR versions on the one hand, and on the other hand the cosine similarity used to measure the textual distance between these versions. Our results show that automatic correction introduces biases in the textual data and that filtering the output NEs seems to be a more promising approach. Finally, we find that incorrect NEs are overwhelmingly short and hapax entities in corpus, and that the entity length and document frequency are the discriminatory criteria to exclude candidate NEs with a precision of over 98%. (shrink)

As a concrete basis for thought, we first share the experience of a correspondence analysis applied to a morphosyntactically tagged corpus. The automatic tagging has been partially checked and shows errors. Knowing how factorial analysis works, and having identified the noisiest tags, we adjust the data to be submitted to the analysis so that it is both reliable and clear, making sense for the interpretation of the results to be drawn.This experiment calls for a three-fold discussion. Firstly, about the evaluation (...) of such a process: indeed, by bypassing (rather than correcting) errors (noise), we shift the analysis framework, which no longer allows an evaluation based on the usual comparison of results and improvement measurement. Next, we justify how tweaking the data can still be a sound scientific approach, when the process is methodical and transparent. Then we consider whether corpus size matters, in relationship with the law of large numbers: statistics neutralize the noise of random fluctuations, but not that of repetitive bias.Thus, we argue for “taming” noise: that is, knowing your corpus and your tools, so as to be able to design quality analyses while managing the residual noise, and freeing yourself from a prior and probably illusory requirement for data perfection. (shrink)

In this article, we describe the various steps that led to the digitization of the dialectological records compiled by Gaston Dulong based on surveys conducted throughout Québec since the late 1940s. Due to their heritage value and scientific interest, these records are currently added to the Fonds de données linguistiques du Québec (FDLQ). However, preparing this corpus poses significant challenges due to the large number of documents to be processed, their physical condition as well as the nature of the data (...) they contain. To ensure data quality, we have favoured an approach that allows for a reasoned way of dealing with the noise caused by computerization. (shrink)

Corpus linguistics has benefited greatly from advances in data processing. However, for the ordinary linguist, the use of morphosyntactic taggers still carries a certain element of mystery and magic. Equipped with this tool, the linguist can work on massive data, but is mostly unaware of how the labels were assigned, and more fundamentally, what errors may have arisen during labeling. And yet, the quality of the labeling process will determine the quality of the results used for analysis. While noise is (...) fairly easy to detect in tagged data, the resulting silence elsewhere is much more insidious, difficult to track down, and therefore potentially more damaging to linguistic analyses. In our study, we compared our manual tagging of Subjunctive Present occurrences in a ten-hour corpus of spoken French with the tagging provided by TreeTagger, TreeTagger-TCOF and Dismo. We'll show that noise in the tagging of the Present tense is the main cause of the high rate of silence in the tagging of the Subjunctive, the confusions being due –as revealed by our exhaustive analysis of the Subjunctive Present forms of the 6500 verbs in the Petit Robert dictionary– to common orthographic forms with the Present tense. Yet the “precision” of taggers is often presented as reaching scores of around 95% (including for spoken French). While “precision” rates here range from 70% to 99%, “recall” rates are much lower: from 44% to 61%. In short, despite its name, the so-called “precision” rate, calculated on all tags, is a poor indicator of the quality of tagging when it comes to a specific label such as SubjPr. (shrink)

While studies on wine tasting are abundant, particularly in sensory marketing, few analyses focus on the wine glass as a specific object. This object has been examined from various perspectives, including alcohol consumption (Banwell 1999), social practices (Lo Monaco et al. 2009), and even more intriguingly, the influence of the glass on the perception of wine aroma (Cliff 2001, Delwiche & Pelchat 2002). The purpose of this article, however, is to to analyze the linguistic and communicational object of the "wine (...) glass" in its discursive contexts, particularly within the digital space, in order to understand the representations associated with it and how the object is staged from a discursive perspective. (shrink)