In recent years, there has been a surge of high-profile publications on applications of artificial intelligence (AI) systems for medical diagnosis and prognosis. While AI provides various opportunities for medical practice, there is an emerging consensus that the existing studies show considerable deficits and are unable to establish the clinical benefit of AI systems. Hence, the view that the clinical benefit of AI systems needs to be studied in clinical trials—particularly randomised controlled trials (RCTs)—is gaining ground. However, an (...) issue that has been overlooked so far in the debate is that, compared with drug RCTs, AI RCTs require methodological adjustments, which entail ethical challenges. This paper sets out to develop a systematic account of the ethics of AI RCTs by focusing on the moral principles of clinical equipoise, informed consent and fairness. This way, the objective is to animate further debate on the (research) ethics of medical AI. (shrink)

There is much to learn from Durán and Jongsma’s paper.1 One particularly important insight concerns the relationship between epistemology and ethics in medical artificial intelligence. In clinical environments, the task of AI systems is to provide risk estimates or diagnostic decisions, which then need to be weighed by physicians. Hence, while the implementation of AI systems might give rise to ethical issues—for example, overtreatment, defensive medicine or paternalism2—the issue that lies at the heart is an epistemic problem: how can (...) physicians know whether to trust decisions made by AI systems? In this manner, various studies examining the interaction of AI systems and physicians have shown that without being able to evaluate their trustworthiness, especially novice physicians become over-reliant on algorithmic support—and ultimately are led astray by incorrect decisions.3–5 This leads to a second insight from the paper, namely that even if some AI system happens to be opaque, it is still not built on the moon. To assess its trustworthiness, AI developers or physicians have different sorts of higher order evidence at hand. Most importantly, …. (shrink)

Forms of Artificial Intelligence (AI) are already being deployed into clinical settings and research into its future healthcare uses is accelerating. Despite this trajectory, more research is needed regarding the impacts on patients of increasing AI decision making. In particular, the impersonal nature of AI means that its deployment in highly sensitive contexts-of-use, such as in healthcare, raises issues associated with patients’ perceptions of (un) dignified treatment. We explore this issue through an experimental vignette study comparing individuals’ perceptions of being (...) treated in a dignified and respectful way in various healthcare decision contexts. Participants were subject to a 2 (human or AI decision maker) x 2 (positive or negative decision outcome) x 2 (diagnostic or resource allocation healthcare scenario) factorial design. We found evidence of a “human bias” (i.e., a preference for human over AI decision makers) and an “outcome bias” (i.e., a preference for positive over negative outcomes). However, we found that for perceptions of respectful and dignified interpersonal treatment, it matters more who makes the decisions in diagnostic cases and it matters more what the outcomes are for resource allocation cases. We also found that humans were consistently viewed as appropriate decision makers and AI was viewed as dehumanizing, and that participants perceived they were treated better when subject to diagnostic as opposed to resource allocation decisions. Thematic coding of open-ended text responses supported these results. We also outline the theoretical and practical implications of these findings. (shrink)

Artificial intelligence (AI) systems have demonstrated impressive performance across a variety of clinical tasks. However, notoriously, sometimes these systems are 'black boxes'. The initial response in the literature was a demand for 'explainable AI'. However, recently, several authors have suggested that making AI more explainable or 'interpretable' is likely to be at the cost of the accuracy of these systems and that prioritising interpretability in medical AI may constitute a 'lethal prejudice'. In this paper, we defend the value of (...) interpretability in the context of the use of AI in medicine. Clinicians may prefer interpretable systems over more accurate black boxes, which in turn is sufficient to give designers of AI reason to prefer more interpretable systems in order to ensure that AI is adopted and its benefits realised. Moreover, clinicians may be justified in this preference. Achieving the downstream benefits from AI is critically dependent on how the outputs of these systems are interpreted by physicians and patients. A preference for the use of highly accurate black box AI systems, over less accurate but more interpretable systems , may itself constitute a form of lethal prejudice that may diminish the benefits of AI to - and perhaps even harm - patients. (shrink)

Various publications claim that medical AI systems perform as well, or better, than clinical experts. However, there have been very few controlled trials and the quality of existing studies has been called into question. There is growing concern that existing studies overestimate the clinical benefits of AI systems. This has led to calls for more, and higher-quality, randomized controlled trials of medical AI systems. While this a welcome development, AI RCTs raise novel methodological challenges that have seen little (...) discussion. We discuss some of the challenges arising in the context of AI RCTs and make some suggestions for how to meet them. (shrink)

Artificial intelligence (AI) systems have demonstrated impressive performance across a variety of clinical tasks. However, notoriously, sometimes these systems are “black boxes.” The initial response in the literature was a demand for “explainable AI.” However, recently, several authors have suggested that making AI more explainable or “interpretable” is likely to be at the cost of the accuracy of these systems and that prioritizing interpretability in medical AI may constitute a “lethal prejudice.” In this paper, we defend the value of (...) interpretability in the context of the use of AI in medicine. Clinicians may prefer interpretable systems over more accurate black boxes, which in turn is sufficient to give designers of AI reason to prefer more interpretable systems in order to ensure that AI is adopted and its benefits realized. Moreover, clinicians may be justified in this preference. Achieving the downstream benefits from AI is critically dependent on how the outputs of these systems are interpreted by physicians and patients. A preference for the use of highly accurate black box AI systems, over less accurate but more interpretable systems, may itself constitute a form of lethal prejudice that may diminish the benefits of AI to—and perhaps even harm—patients. (shrink)

In this article, we explore questions about the culture of trustworthy artificial intelligence (AI) through the lens of ecosystems. We draw on the European Commission’s Guidelines for Trustworthy AI and its philosophical underpinnings. Based on the latter, the trustworthiness of an AI ecosystem can be conceived of as being grounded by both the so-called rational-choice and motivation-attributing accounts—i.e., trusting is rational because solution providers deliver expected services reliably, while trust also involves resigning control by attributing one’s motivation, and hence, goals, (...) onto another entity. Our research question is: What aspects contribute to a responsible AI ecosystem that can promote justifiable trustworthiness in a healthcare environment? We argue that especially within devising governance and support aspects of a medical AI ecosystem, considering the so-called motivation-attributing account of trust provides fruitful pointers. There can and should be specific ways and governance structures supporting and nurturing trustworthiness beyond mere reliability. After compiling a list of preliminary requirements for this, we describe the emergence of one particular medical AI ecosystem and assess its compliance with and future ways of improving its functioning as a responsible AI ecosystem that promotes trustworthiness. (shrink)

Artificial intelligence (AI) is expected to revolutionise the practice of medicine. Recent advancements in the field of deep learning have demonstrated success in variety of clinical tasks: detecting diabetic retinopathy from images, predicting hospital readmissions, aiding in the discovery of new drugs, etc. AI’s progress in medicine, however, has led to concerns regarding the potential effects of this technology on relationships of trust in clinical practice. In this paper, I will argue that there is merit to these concerns, since AI (...) systems can be relied on, and are capable of reliability, but cannot be trusted, and are not capable of trustworthiness. Insofar as patients are required to rely on AI systems for their medical decision-making, there is potential for this to produce a deficit of trust in relationships in clinical practice. (shrink)

Medical artificial intelligence (AI) systems are value-laden technologies that can simultaneously encourage and discourage conflicting values that may all be relevant for the pursuit of justice. I argue that the predominant theory of healthcare justice, the Rawls-inspired approach of Norman Daniels, neither adequately acknowledges such conflicts nor explains if and how they can resolved. By juxtaposing Daniels’s theory of healthcare justice with Axel Honneth’s and Nancy Fraser’s respective theories of justice, I draw attention to one such conflict. Medical (...) AI may improve the distribution of opportunity qua health while simultaneously mis-recognizing patients and thereby reducing their self-respect. I argue that justly resolving this conflict will at times require greater inclusion of those mis-recognized in deliberation about medical AI, and consider what such inclusion may entail. (shrink)

:This article considers recent ethical topics relating to medical AI. After a general discussion of recent medical AI innovations, and a more analytic look at related ethical issues such as data privacy, physician dependency on poorly understood AI helpware, bias in data used to create algorithms post-GDPR, and changes to the patient–physician relationship, the article examines the issue of so-called robot doctors. Whereas the so-called democratization of healthcare due to health wearables and increased access to medical information (...) might suggest a positive shift in the patient-physician relationship, the physician’s ‘need to care’ might be irreplaceable, and robot healthcare workers might be seen as contributing to dehumanized healthcare practices. (shrink)

Some patients, following brain injury, do not outwardly respond to spoken commands, yet show patterns of brain activity that indicate responsiveness. This is “cognitive-motor dissociation” (CMD). Recent research has used machine learning to diagnose CMD from electroencephalogram (EEG) recordings. These techniques have high false discovery rates, raising a serious problem of inductive risk. It is no solution to communicate the false discovery rates directly to the patient’s family, because this information may confuse, alarm and mislead. Instead, we need a procedure (...) for generating case-specific probabilistic assessments that can be communicated clearly. This article constructs a possible procedure with three key elements: (1) a shift from categorical “responding or not” assessments to degrees of evidence; (2) the use of patient-centred priors to convert degrees of evidence to probabilistic assessments; and (3) the use of standardized probability yardsticks to convey those assessments as clearly as possible. (shrink)

The use of black box algorithms in medicine has raised scholarly concerns due to their opaqueness and lack of trustworthiness. Concerns about potential bias, accountability and responsibility, patient autonomy and compromised trust transpire with black box algorithms. These worries connect epistemic concerns with normative issues. In this paper, we outline that black box algorithms are less problematic for epistemic reasons than many scholars seem to believe. By outlining that more transparency in algorithms is not always necessary, and by explaining that (...) computational processes are indeed methodologically opaque to humans, we argue that the reliability of algorithms provides reasons for trusting the outcomes of medical artificial intelligence. To this end, we explain how computational reliabilism, which does not require transparency and supports the reliability of algorithms, justifies the belief that results of medical AI are to be trusted. We also argue that several ethical concerns remain with black box algorithms, even when the results are trustworthy. Having justified knowledge from reliable indicators is, therefore, necessary but not sufficient for normatively justifying physicians to act. This means that deliberation about the results of reliable algorithms is required to find out what is a desirable action. Thus understood, we argue that such challenges should not dismiss the use of black box algorithms altogether but should inform the way in which these algorithms are designed and implemented. When physicians are trained to acquire the necessary skills and expertise, and collaborate with medical informatics and data scientists, black box algorithms can contribute to improving medical care. (shrink)

Ugar and Malele write that generic machine learning (ML) technologies for mental health diagnosis would be challenging to implement in sub-Saharan Africa due to cultural specificities in how those conditions are diagnosed. For example, they say that in South Africa, the appearance of ‘schizophrenia’ might be understood as a type of spiritual possession, rather than a mental disorder caused by a brain dysfunction. Hence, a generic ML system is likely to ‘misdiagnose’ persons whose symptomatology matches that of schizophrenia in the (...) South African context. The authors thus claim that ‘a generic or universal design cannot be effective given the heterogeneity of value judgements in defining what mental health disorders are in different contexts’.1 Should we take this to mean that ML systems ‘should not be designed with a generalised perception of mental disorders,’1 as the authors suggest? On the contrary, my view is that generic ML can be useful, with the caveat that issues of cultural sensitivity may engender translational challenges in various national and cultural contexts. Yet, if it can be demonstrated that a generic ML system can reliably and accurately pick up on shared symptomatologies across cultures despite what any individual or community might believe about their actual causes, this aspect is surely the function and value of diagnostic ML which we should take to matter. Cultural gaps in aetiological understandings of mental health conditions make it all the more important to advance generic ML as a calibrating diagnostic tool. In this commentary, therefore, I will endeavour to make a case for generic ML in mental healthcare. ### Aetiological (mis)understanding across cultures The fundamental complexity to which Ugar and Malele direct our attention is the fact that mental health conditions are attributed to variable causal explanations, rather than a …. (shrink)

In the past few years, machine learning (ML) tools have been implemented with success in the medical context. However, several practitioners have raised concerns about the lack of transparency—at the algorithmic level—of many of these tools; and solutions from the field of explainable AI (XAI) have been seen as a way to open the ‘black box’ and make the tools more trustworthy. Recently, Alex London has argued that in the medical context we do not need machine learning tools (...) to be interpretable at the algorithmic level to make them trustworthy, as long as they meet some strict empirical desiderata. In this paper, we analyse and develop London’s position. In particular, we make two claims. First, we claim that London’s solution to the problem of trust can potentially address another problem, which is how to evaluate the reliability of ML tools in medicine for regulatory purposes. Second, we claim that to deal with this problem, we need to develop London’s views by shifting the focus from the opacity of algorithmic details to the opacity of the way in which ML tools are trained and built. We claim that to regulate AI tools and evaluate their reliability, agencies need an explanation of how ML tools have been built, which requires documenting and justifying the technical choices that practitioners have made in designing such tools. This is because different algorithmic designs may lead to different outcomes, and to the realization of different purposes. However, given that technical choices underlying algorithmic design are shaped by value-laden considerations, opening the black box of the design process means also making transparent and motivating (technical and ethical) values and preferences behind such choices. Using tools from philosophy of technology and philosophy of science, we elaborate a framework showing how an explanation of the training processes of ML tools in medicine should look like. (shrink)

Artificial intelligence (AI) is increasingly relied upon by clinicians for making diagnostic and treatment decisions, playing an important role in imaging, diagnosis, risk analysis, lifestyle monitoring, and health information management. While research has identified biases in healthcare AI systems and proposed technical solutions to address these, we argue that effective solutions require human engagement. Furthermore, there is a lack of research on how to motivate the adoption of these solutions and promote investment in designing AI systems that align with values (...) such as transparency and fairness from the outset. Drawing on insights from psychological theories, we assert the need to understand the values that underlie decisions made by individuals involved in creating and deploying AI systems. We describe how this understanding can be leveraged to increase engagement with de-biasing and fairness-enhancing practices within the AI healthcare industry, ultimately leading to sustained behavioral change via autonomy-supportive communication strategies rooted in motivational and social psychology theories. In developing these pathways to engagement, we consider the norms and needs that govern the AI healthcare domain, and we evaluate incentives for maintaining the status quo against economic, legal, and social incentives for behavior change in line with transparency and fairness values. (shrink)

In his recent article ‘Limits of trust in medical AI,’ Hatherley argues that, if we believe that the motivations that are usually recognised as relevant for interpersonal trust have to be applied to interactions between humans and medical artificial intelligence, then these systems do not appear to be the appropriate objects of trust. In this response, we argue that it is possible to discuss trust in medical artificial intelligence (AI), if one refrains from simply assuming that trust (...) describes human–human interactions. To do so, we consider an account of trust that distinguishes trust from reliance in a way that is compatible with trusting non-human agents. In this account, to trust a medical AI is to rely on it with little monitoring and control of the elements that make it trustworthy. This attitude does not imply specific properties in the AI system that in fact only humans can have. This account of trust is applicable, in particular, to all cases where a physician relies on the medical AI predictions to support his or her decision making. (shrink)

The emergence of ethical concerns surrounding artificial intelligence (AI) has led to an explosion of high-level ethical principles being published by a wide range of public and private organizations. However, there is a need to consider how AI developers can be practically assisted to anticipate, identify and address ethical issues regarding AI technologies. This is particularly important in the development of AI intended for healthcare settings, where applications will often interact directly with patients in various states of vulnerability. In this (...) paper, we propose that an ‘embedded ethics’ approach, in which ethicists and developers together address ethical issues via an iterative and continuous process from the outset of development, could be an effective means of integrating robust ethical considerations into the practical development of medical AI. (shrink)

Bioethics, Volume 36, Issue 2, Page 121-133, February 2022.

This is the first book to provide a coherent overview over the ethical implications of AI-related technologies in medicine. It explores how these technologies transform practices, relationships, and environments in the clinical field. It provides an introduction into ethical issues such as data security and privacy protection, bias and algorithmic fairness, trust and transparency, challenges to the doctor-patient relationship, and new perspectives for informed consent. The book focuses on the transformative impact that technology is having on medicine, and discusses several (...) strategies for dealing with the resulting challenges. It also introduces innovative methods of ethics research for addressing existing desiderata and future challenges. This book is written to inform health care professionals, policy-makers, and researchers in medicine, health sciences, nursing science, social sciences, and ethics, but may also function as a primary textbook for graduate as well as undergraduate university courses. (shrink)

In the years to come, artificial intelligence will become an indispensable tool in medical practice. The digital transformation will undoubtedly affect today’s medical students. This study focuses on trust from the perspective of three groups of medical students - students from Croatia, students from Slovakia, and international students studying in Slovakia. A paper-pen survey was conducted using a non-probabilistic convenience sample. In the second half of 2022, 1715 students were surveyed at five faculties in Croatia and three (...) in Slovakia. Specifically, 38.2% of students indicated familiarity with the concept of AI, while 44.8% believed they would use AI in the future. Patient readiness for the implementation of technologies was mostly assessed as being low. More than half of the students, 59.1%, believe that the implementation of digital technology (AI) will negatively impact the patient-physician relationship and 51,3% of students believe that patients will trust physicians less. The least agreement with the statement was observed among international students, while a higher agreement was expressed by Slovak and Croatian students 40.9% of Croatian students believe that users do not trust the healthcare system, 56.9% of Slovak students agree with this view, while only 17.3% of international students share this opinion. The ability to explain to patients how AI works if they were asked was statistically significantly different for the different student groups, international students expressed the lowest agreement, while the Slovak and Croatian students showed a higher agreement. This study provides insight into medical students’ attitudes from Croatia, Slovakia, and international students regarding the role of artificial intelligence (AI) in the future healthcare system, with a particular emphasis on the concept of trust. A notable difference was observed between the three groups of students, with international students differing from their Croatian and Slovak colleagues. This study also highlights the importance of integrating AI topics into the medical curriculum, taking into account national social & cultural specificities that could negatively impact AI implementation if not carefully addressed. (shrink)

This paper concerns the double standard debate in the ethics of AI literature. This debate essentially revolves around the question of whether we should subject AI systems to different normative standards than humans. So far, the debate has centered around the desideratum of transparency. That is, the debate has focused on whether AI systems must be more transparent than humans in their decision-making processes in order for it to be morally permissible to use such systems. Some have argued that the (...) same standards of transparency should be applied across the board, for AI systems and humans alike. Others have argued that we should hold AI systems to higher standards than humans in terms of transparency. In this paper, we first point out that there are structurally similar double standard debates to be had about other desiderata besides transparency, such as predictive accuracy. Second, we argue that when we focus on predictive accuracy, there are at least two reasons for holding AI systems to a lower standard than humans. (shrink)

The increased presence of medical AI in clinical use raises the ethical question which standard of explainability is required for an acceptable and responsible implementation of AI-based applications in medical contexts. In this paper, we elaborate on the emerging debate surrounding the standards of explainability for medical AI. For this, we first distinguish several goods explainability is usually considered to contribute to the use of AI in general, and medical AI in specific. Second, we propose to (...) understand the value of explainability relative to other available norms of explainable decision-making. Third, in pointing out that we usually accept heuristics and uses of bounded rationality for medical decision-making by physicians, we argue that the explainability of medical decisions should not be measured against an idealized diagnostic process, but according to practical considerations. We conclude, fourth, to resolve the issue of explainability-standards by relocating the issue to the AI’s certifiability and interpretability. (shrink)

The escalating integration of Artificial Intelligence (AI) in clinical settings carries profound implications for the doctrine of informed consent, presenting challenges that necessitate immediate attention. China, in its advancement in the deployment of medical AI, is proactively engaging in the formulation of legal and ethical regulations. This paper takes China as an example to undertake a theoretical examination rooted in the principles of medical ethics and legal norms, analyzing informed consent and medical AI through relevant literature data. (...) The study reveals that medical AI poses fundamental challenges to the accuracy, adequacy, and objectivity of information disclosed by doctors, alongside impacting patient competency and willingness to give consent. To enhance adherence to informed consent rules in the context of medical AI, this paper advocates for a shift towards a patient‐centric information disclosure standard, the restructuring of medical liability rules, the augmentation of professional training, and the advancement of public understanding through educational initiatives. (shrink)

Bioethics, Volume 36, Issue 2, Page 162-169, February 2022.

Artificial intelligence is changing healthcare and the practice of medicine as data-driven science and machine-learning technologies, in particular, are contributing to a variety of medical and clinical tasks. Such advancements have also raised many questions, especially about public trust. As a response to these concerns there has been a concentrated effort from public bodies, policy-makers and technology companies leading the way in AI to address what is identified as a "public trust deficit". This paper argues that a focus on (...) trust as the basis upon which a relationship between this new technology and the public is built is, at best, ineffective, at worst, inappropriate or even dangerous, as it diverts attention from what is actually needed to actively warrant trust. Instead of agonising about how to facilitate trust, a type of relationship which can leave those trusting vulnerable and exposed, we argue that efforts should be focused on the difficult and dynamic process of ensuring reliance underwritten by strong legal and regulatory frameworks. From there, trust could emerge but not merely as a means to an end. Instead, as something to work in practice towards; that is, the deserved result of an ongoing ethical relationship where there is the appropriate, enforceable and reliable regulatory infrastructure in place for problems, challenges and power asymmetries to be continuously accounted for and appropriately redressed. Data sharing not applicable as no datasets generated and/or analysed for this study. (shrink)

In their article ‘Who is afraid of black box algorithms? On the epistemological and ethical basis of trust in medical AI’, Durán and Jongsma discuss the epistemic and ethical challenges raised by black box algorithms in medical practice. The opacity of black box algorithms is an obstacle to the trustworthiness of their outcomes. Moreover, the use of opaque algorithms is not normatively justified in medical practice. The authors introduce a formalism, called computational reliabilism, which allows generating justified (...) beliefs on the algorithm reliability and trustworthy outcomes of artificial intelligence systems by means of epistemic warrants, called reliability indicators. However, they remark the need for reliability indicators specific to black box algorithms and that justified knowledge is not sufficient to justify normatively the actions of the physicians using medical AI systems. Therefore, Durán and Jongsma advocate for a more transparent design and implementation of black box algorithms, providing a series of recommendations to mitigate the epistemic and ethical challenges behind their use in medical practice. In this response, I argue that a peculiar form of black box algorithm transparency, called design publicity, may efficiently implement these recommendations. Design publicity encodes epistemic, that is, reliability indicators, and ethical recommendations for black box algorithms by means of four subtypes of transparency. These target the values and goals, their translation into design requirements, the performance and consistency of the algorithm altogether. I discuss design publicity applying it to a use case focused on the automated classification of skin lesions from medical images. (shrink)

Artificial intelligence (AI) is increasingly being developed for use in medicine, including for diagnosis and in treatment decision making. The use of AI in medical treatment raises many ethical issues that are yet to be explored in depth by bioethicists. In this paper, I focus specifically on the relationship between the ethical ideal of shared decision making and AI systems that generate treatment recommendations, using the example of IBM’s Watson for Oncology. I argue that use of this type of (...) system creates both important risks and significant opportunities for promoting shared decision making. If value judgements are fixed and covert in AI systems, then we risk a shift back to more paternalistic medical care. However, if designed and used in an ethically informed way, AI could offer a potentially powerful way of supporting shared decision making. It could be used to incorporate explicit value reflection, promoting patient autonomy. In the context of medical treatment, we need value-flexible AI that can both respond to the values and treatment goals of individual patients and support clinicians to engage in shared decision making. (shrink)

In this paper, two central questions will be addressed: ought we to implement medical AI technology in the medical domain? If yes, how ought we to implement this technology? I will critically engage with three options that exist with respect to these central questions: the Neo-Luddite option, the Assistive option, and the Substitutive option. I will first address key objections on behalf of the Neo-Luddite option: the Objection from Bias, the Objection from Artificial Autonomy, the Objection from Status (...) Quo, and the Objection from Inscrutability. I will thereafter present the Demographic Trends Argument and the Human Enhancement Argument in support of alternatives to the Neo-Luddite option. In the second half of the paper, I will argue against the Substitutive option and in favour of the Assistive option, given the existence of two chief formal deficits in medical AI technology: the causality deficit and the care deficit. (shrink)

Explainable artificial intelligence (XAI) is an emerging, multidisciplinary field of research that seeks to develop methods and tools for making AI systems more explainable or interpretable. XAI researchers increasingly recognise explainability as a context-, audience- and purpose-sensitive phenomenon, rather than a single well-defined property that can be directly measured and optimised. However, since there is currently no overarching definition of explainability, this poses a risk of miscommunication between the many different researchers within this multidisciplinary space. This is the problem we (...) seek to address in this paper. We outline a framework, called Explanatory Pragmatism, which we argue has two attractive features. First, it allows us to conceptualise explainability in explicitly context-, audience- and purpose-relative terms, while retaining a unified underlying definition of explainability. Second, it makes visible any normative disagreements that may underpin conflicting claims about explainability regarding the purposes for which explanations are sought. Third, it allows us to distinguish several dimensions of AI explainability. We illustrate this framework by applying it to a case study involving a machine learning model for predicting whether patients suffering disorders of consciousness were likely to recover consciousness. (shrink)

Volume 25, Issue 3, March 2025, Page 132-135.

I analyse an argument according to which medical artificial intelligence represents a threat to patient autonomy—recently put forward by Rosalind McDougall in the Journal of Medical Ethics. The argument takes the case of IBM Watson for Oncology to argue that such technologies risk disregarding the individual values and wishes of patients. I find three problems with this argument: it confuses AI with machine learning; it misses machine learning’s potential for personalised medicine through big data; it fails to distinguish (...) between evidence-based advice and decision-making within healthcare. I conclude that how much and which tasks we should delegate to machine learning and other technologies within healthcare and beyond is indeed a crucial question of our time, but in order to answer it, we must be careful in analysing and properly distinguish between the different systems and different delegated tasks. (shrink)

Cohen’s (2023) mapping exercise of possible bioethical issues emerging from the use of ChatGPT in medicine provides an informative, useful, and thought-provoking trigger for discussions of AI ethic...

This paper considers the ethical risks and opportunities presented by generalist medical artificial intelligence (GMAI), a kind of dynamic, multimodal AI proposed by Moor et al. (2023) for use in health care. The research objective is to apply widely accepted principles of biomedical ethics to analyze the possible consequences of GMAI, while emphasizing the distinctions between GMAI and current-generation, task-specific medical AI. The principles of autonomy and health equity in particular provide useful guidance for the ethical risks and (...) opportunities of novel AI systems in health care. The ethics of two applications of GMAI are examined: enabling decision aids that inform and educate patients about certain treatments and conditions, and expanding AI-driven diagnosis and treatment recommendation. Emphasis is placed on the potential of GMAI to improve shared decision-making between patients and providers, which supports patient autonomy. Another focus is on health equity, or the reduction of health and access disparities facing underserved populations. Although GMAI presents opportunities to improve patient autonomy, health literacy, and health equity, premature or inadequately regulated adoption of GMAI has the potential to compromise both health equity and patient autonomy. On the other hand, there are significant risks to health equity and autonomy that may arise from not adopting GMAI that has been thoroughly validated and tested. A careful balancing of these risks and benefits will be required to secure the best ethical outcome, if GMAI is ever employed at scale. (shrink)

This paper conducts a comparative analysis of data governance mechanisms concerning the secondary use of health data in Taiwan and the European Union (EU). Both regions have adopted distinctive approaches and regulations for utilizing health data beyond primary care, encompassing areas such as medical research and healthcare system enhancement. Through an examination of these models, this study seeks to elucidate the strategies, frameworks, and legal structures employed by Taiwan and the EU to strike a delicate balance between the imperative (...) of data-driven healthcare innovation and the safeguarding of individual privacy rights. This paper examines and compares several key aspects of the secondary use of health data in Taiwan and the EU. These aspects include data governance frameworks, legal and regulatory frameworks, data access and sharing mechanisms, and privacy and security considerations. This comparative exploration offers invaluable insights into the evolving global landscape of health data governance. It provides a deeper understanding of the strategies implemented by these regions to harness the potential of health data while upholding the ethical and legal considerations surrounding its secondary use. The findings aim to inform best practices for responsible and effective health data utilization, particularly in the context of medical AI applications. (shrink)

In this paper, we argue that one way to approach what is known in the literature as the “Trust Gap” in Medical AI is to focus on explanations from an Explainable AI (xAI) perspective. Against the current framework on xAI – which does not offer a real solution – we argue for a pragmatist turn, one that focuses on understanding how we provide explanations in Traditional Medicine (TM), composed by human agents only. Following this, explanations have two specific relevant (...) components: they are usually (i) social and (ii) abductive. Explanations, in this sense, ought to provide understanding by answering contrastive _why-questions_: “Why had P happened instead of Q?” (Miller in AI 267:1–38, 2019) (Sect. 1 ). In order to test the relevancy of this concept of explanation in medical xAI, we offer several reasons to argue that abductions are crucial for medical reasoning and provide a crucial tool to deal with trust gaps between human agents (Sect. 2 ). If abductions are relevant in TM, we can test the capability of Artificial Intelligence systems on this merit. Therefore, we provide an analysis of the capacity for social and abductive reasoning of different AI technologies. Accordingly, we posit that Large Language Models (LLMs) and transformer architectures exhibit a noteworthy potential for effective engagement in abductive reasoning. By leveraging the potential abductive capabilities of LLMs and transformers, we anticipate a paradigm shift in the integration of explanations within AI systems. This, in turn, has the potential to enhance the trustworthiness of AI-driven medical decisions, bridging the Trust Gap that has been a prominent challenge in the field of Medical AI (Sect. 3 ). This development holds the potential to not only improve the interpretability of AI-generated medical insights but also to guarantee that trust among practitioners, patients, and stakeholders in the healthcare domain is still present. (shrink)

There is a current debate about if, and in what sense, machine learning systems used in the medical context need to be explainable. Those arguing in favor contend these systems require post hoc explanations for each individual decision to increase trust and ensure accurate diagnoses. Those arguing against suggest the high accuracy and reliability of the systems is sufficient for providing epistemic justified beliefs without the need for explaining each individual decision. But, as we show, both solutions have limitations—and (...) it is unclear either address the epistemic worries of the medical professionals using these systems. We argue these systems do require an explanation, but an institutional explanation. These types of explanations provide the reasons why the medical professional should rely on the system in practice—that is, they focus on trying to address the epistemic concerns of those using the system in specific contexts and specific occasions. But ensuring that these institutional explanations are fit for purpose means ensuring the institutions designing and deploying these systems are transparent about the assumptions baked into the system. This requires coordination with experts and end-users concerning how it will function in the field, the metrics used to evaluate its accuracy, and the procedures for auditing the system to prevent biases and failures from going unaddressed. We contend this broader explanation is necessary for either post hoc explanations or accuracy scores to be epistemically meaningful to the medical professional, making it possible for them to rely on these systems as effective and useful tools in their practices. (shrink)

Bioethics, Volume 36, Issue 2, Page 111-112, February 2022.

In this article I consider two pertinent questions that practitioners must consider when they deploy an algorithmic system as support in clinical shared decision‐making. The first question concerns how to interpret and assess the significance of different performance measures for clinical decision‐making. The second question concerns the professional obligations that practitioners have to communicate information about the quality of an algorithm's output to patients in light of the principles of autonomy, beneficence, and justice. In the article I review the four (...) standard performance measures used to evaluate and validate algorithms, outline their role in the discussion of algorithmic fairness, and discuss the professional responsibilities that practitioners face when communicating information about these measures to patients. (shrink)

Participation in research is supposed to be voluntary and informed. Yet it is difficult to ensure people are adequately informed about the potential uses of their biological materials when they donate samples for future research. We propose a novel consent framework which we call “demonstrated consent” that leverages blockchain technology and generative AI to address this problem. In a demonstrated consent model, each donated sample is associated with a unique non-fungible token (NFT) on a blockchain, which records in its metadata (...) information about the planned and past uses of the sample in research, and is updated with each use of the sample. This information is accessible to a large language model (LLM) customized to present this information in an understandable and interactive manner. Thus, our model uses blockchain and generative AI technologies to track, make available, and explain information regarding planned and past uses of donated samples. (shrink)

The original version of this article unfortunately contains an unconverted data in footnotes 5, 9 and 13.

In contrast to Di Nucci’s characterisation, my argument is not a technoapocalyptic one. The view I put forward is that systems like IBM’s Watson for Oncology create both risks and opportunities from the perspective of shared decision-making. In this response, I address the issues that Di Nucci raises and highlight the importance of bioethicists engaging critically with these developing technologies.

In their article “What are Humans Doing in the Loop? Co-Reasoning and Practical Judgment When Using Machine Learning-Driven Decision Aids,” Salloch and Eriksen (2024) propose involving patients as...

With the growing application of machine learning models in medicine, principlist bioethics has been put forward as needing revision. This paper reflects on the dominant trope in AI ethics to include a new ‘principle of explicability’ alongside the traditional four principles of bioethics that make up the theory of principlism. It specifically suggests that these four principles are sufficient and challenges the relevance of explicability as a separate ethical principle by emphasizing the coherentist affinity of principlism. We argue that, through (...) specification, the properties of explicability are already covered by the four bioethical principles. The paper finishes by anticipating an objection that coherent principles could not facilitate technology induced change and are not well-suited to tackle moral differences. (shrink)

The paper by Salloch and Eriksen (2024) offers an insightful contribution to the ethical debate on Machine Learning-driven Clinical Decision Support Systems (ML_CDSS) and provides much-needed conce...

This paper examines the Saudi Food and Drug Authority’s (SFDA) Guidance on Artificial Intelligence (AI) and Machine Learning (ML) technologies based Medical Devices (the MDS-G010). The SFDA has pioneered binding requirements designed for manufacturers to obtain Medical Device Marketing Authorization. The regulation of AI in health is at an early stage worldwide. Therefore, it is critical to examine the scope and nature of the MDS-G010, its influences, and its future directions. It is argued that the guidance is a (...) patchwork of existing international best practices concerning AI regulation, incorporates adapted forms of non-AI-based guidelines, and builds on existing legal requirements in the SFDA’s existing regulatory architecture. There is particular congruence with the approaches of the US Food and Drug Administration (FDA) and the International Medical Device Regulators Forum (IMDRF), but the SFDA goes beyond those approaches to incorporate other best practices into its guidance. Additionally, the binding nature of the MDS-G010 is complex. There are binding ‘components’ within the guidance, but the incorporation of non-binding international best practices which are subordinate to national law results in a lack of clarity about how penalties for non-compliance will operate. (shrink)

Background Caring behaviour is critical for nursing quality, and the clinical internship environment is a crucial setting for preparing nursing students for caring behaviours. Evidence about how to develop nursing students’ caring behaviour in the clinical environment is still emerging. However, the mechanism between the clinical internship environment and caring behaviour remains unclear, especially the mediating role of moral sensitivity and the moderating effect of self-efficacy. Research objective This study aimed to examine the mediating effect of moral sensitivity and the (...) moderating function of self-efficacy on the association between the clinical internship environment and caring behaviours. Research design A cross-sectional design used acceptable validity scales. The hypothesised moderated mediation model was tested in the SPSS PROCESS macro. Participants and research context This survey collected data from 504 nursing students in an internship at a teaching hospital in Changsha, China. Ethical considerations This study was pre-approved by the ethics committee of the medical school (No. E2022210). Informed consent was obtained from all students. Results The clinical internship environment ( B = 0.450, 95% CI = [0.371, 0.530]) and moral sensitivity ( B = 1.352, 95% CI = [1.090, 1.615]) had positive direct effects on nursing students' caring behaviours. Clinical internship environment also indirectly influenced students’ caring behaviours via moral sensitivity ( B = 0.161, 95% CI = [0.115, 0.206]). In addition, self-efficacy played a moderating role between the clinical internship environment and caring behaviours ( B = 0.019, 95% CI = [0.007, 0.031]), as well as the relationship between the clinical internship environment and moral sensitivity ( B = 0.006, 95% CI = [0.003, 0.010]). Conclusion Moral sensitivity mediates the effect of the clinical internship environment on caring behaviour, and self-efficacy strengthens both direct and indirect effects. This study emphasises the importance of self-efficacy in developing moral sensitivity and caring behaviours in nursing students. (shrink)

Since their public launch, a little over a year ago, large language models (LLMs) have inspired a flurry of analysis about what their implications might be for medical ethics, and for society more broadly. 1 Much of the recent debate has moved beyond categorical evaluations of the permissibility or impermissibility of LLM use in different general contexts (eg, at work or school), to more fine-grained discussions of the criteria that should govern their appropriate use in specific domains or towards (...) certain ends. 2 With each passing week, it seems more and more inevitable that LLMs will be a pervasive feature of many, if not most, of our lives. It would not be possible—and would not be desirable—to prohibit them across the board. We need to learn how to live with LLMs; to identify and mitigate the risks they pose to us, to our fellow creatures, and the... (shrink)