Despite recent efforts by the Artificial Intelligence (AI) community to move towards standardised procedures for documenting models, methods, systems or datasets, there is currently no methodology focused on use cases aligned with the risk-based approach of the European AI Act (AI Act). In this paper, we propose a new framework for the documentation of use cases that we call use case cards, based on the use case modelling included in the Unified Markup Language (UML) standard. Unlike other documentation methodologies, we (...) focus on the intended purpose and operational use of an AI system. It consists of two main parts: firstly, a UML-based template, tailored to allow implicitly assessing the risk level of the AI system and defining relevant requirements, and secondly, a supporting UML diagram designed to provide information about the system-user interactions and relationships. The proposed framework is the result of a co-design process involving a relevant team of EU policy experts and scientists. We have validated our proposal with 11 experts with different backgrounds and a reasonable knowledge of the AI Act as a prerequisite. We provide the 5 use case cards used in the co-design and validation process. Use case cards allows framing and contextualising use cases in an effective way, and we hope this methodology can be a useful tool for policy makers and providers for documenting use cases, assessing the risk level, adapting the different requirements and building a catalogue of existing usages of AI. (shrink)

The European Union’s Artificial Intelligence Act (AI Act) is a groundbreaking regulatory framework that integrates technical concepts and terminology from the rapidly evolving ecosystems of AI research and innovation into the legal domain. Precise definitions accessible to both AI experts and lawyers are crucial for the legislation to be effective. This paper provides an interdisciplinary analysis of the concepts of AI system, general purpose AI system, foundation model and generative AI across the different versions of the legal text (Commission proposal, (...) Parliament position and Council General Approach) before the final political agreement. The goal is to help bridge the understanding of these key terms between the technical and legal communities and contribute to a proper implementation of the AI Act. We provide an analysis of the concept of AI system considering its scientific foundation and the crucial role that it plays in the regulation, which requires a sound definition both from legal and technical standpoints. We connect the outcomes of this discussion with the analysis of the concept of general purpose AI system and its evolution during the negotiations. We also address the distinct conceptual meanings of AI system vs AI model and explore the technical nuances of the term foundation model. We conclude that rooting the definition of foundation model to its general purpose capabilities following standardised evaluation methodologies appears to be most appropriate approach. Lastly, we tackle the concept of generative AI, arguing that definitions of AI system that include “content” as one of the system’s outputs already captures it, and concluding that not all generative AI is based on foundation models. (shrink)

In this paper we investigate risk prediction of criminal re-offense among juvenile defendants using general-purpose machine learning algorithms. We show that in our dataset, containing hundreds of cases, ML models achieve better predictive power than a structured professional risk assessment tool, the Structured Assessment of Violence Risk in Youth, at the expense of not satisfying relevant group fairness metrics that SAVRY does satisfy. We explore in more detail two possible causes of this algorithmic bias that are related to biases in (...) the data with respect to two protected groups, foreigners and women. In particular, we look at the differences in the prevalence of re-offense between protected groups and the influence of protected group or correlated features in the prediction. Our experiments show that both can lead to disparity between groups on the considered group fairness metrics. We observe that methods to mitigate the influence of either cause do not guarantee fair outcomes. An analysis of feature importance using LIME, a machine learning interpretability method, shows that some mitigation methods can shift the set of features that ML techniques rely on away from demographics and criminal history which are highly correlated with sensitive features. (shrink)