A morally acceptable course of AI development should avoid two dangers: creating unaligned AI systems that pose a threat to humanity and mistreating AI systems that merit moral consideration in their own right. This paper argues these two dangers interact and that if we create AI systems that merit moral consideration, simultaneously avoiding both of these dangers would be extremely challenging. While our argument is straightforward and supported by a wide range of pretheoretical moral judgments, it has far-reaching moral implications (...) for AI development. Although the most obvious way to avoid the tension between alignment and ethical treatment would be to avoid creating AI systems that merit moral consideration, this option may be unrealistic and is perhaps fleeting. So, we conclude by offering some suggestions for other ways of mitigating mistreatment risks associated with alignment. (shrink)

Roughly, empirical AI alignment research (AIA) is an area of AI research which investigates empirically how to design AI systems in line with human goals. This paper examines the role of non-epistemic values in AIA. It argues that: (1) Sciences differ in the degree to which values influence them. (2) AIA is strongly value-laden. (3) This influence of values is managed inappropriately and thus threatens AIA’s epistemic integrity and ethical beneficence. (4) AIA should strive to achieve value transparency, critical (...) scrutiny from inside and outside the discipline – involving the public –, and to empower actors without strong commercial interests. (shrink)

New AI technologies have the potential to cause unintended harms in diverse domains including warfare, judicial sentencing, biomedicine and governance. One strategy for realising the benefits of AI whilst avoiding its potential dangers is to ensure that new AIs are properly ‘aligned’ with some form of ‘alignment target.’ One danger of this strategy is that – dependent on the alignment target chosen – our AIs might optimise for objectives that reflect the values only of a certain subset of (...) society, and that do not take into account alternative views about what constitutes desirable and safe behaviour for AI agents. In response to this problem, several AI ethicists have suggested alignment targets that are designed to be sensitive to widespread normative disagreement amongst the relevant stakeholders. Authors inspired by voting theory have suggested that AIs should be aligned with the verdicts of actual or simulated ‘moral parliaments’ whose members represent the normative views of the relevant stakeholders. Other authors inspired by decision theory and the philosophical literature on moral uncertainty have suggested that AIs should maximise socially expected choiceworthiness. In this paper, I argue that both of these proposals face several important problems. In particular, they fail to select attractive ‘compromise options’ in cases where such options are available. I go on to propose and defend an alternative, bargaining-theoretic alignment target, which avoids the problems associated with the voting- and decision-theoretic approaches. (shrink)

The AI and AI alignment communities have been instrumental in addressing existential risks, developing alignment methodologies, and promoting rationalist problem-solving approaches. However, as AI research ventures into increasingly uncertain domains, there is a risk of premature epistemic convergence, where prevailing methodologies influence not only the evaluation of ideas but also determine which ideas are considered within the discourse. This paper examines critical epistemic blind spots in AI alignment research, particularly the lack of predictive frameworks to differentiate problems (...) necessitating general intelligence, decentralized intelligence, or innovative alignment methodologies. We analyze how heuristic-based epistemic filtering—favoring ideas that conform to established norms over those assessed purely on logical merit—may inadvertently constrain the field's potential for groundbreaking solutions. The semblance of rigor, wherein social consensus and stylistic conformity replace foundational reasoning, further intensifies this concern. To mitigate these issues, we advocate for an epistemically adaptive research environment that emphasizes structured evaluation of paradigm-challenging ideas, cross-disciplinary collaboration, and strategies to identify and reduce epistemic overfitting. By broadening the epistemic frameworks governing AI alignment discourse, the field can progress beyond incremental improvements toward discovering entirely new solution paradigms. Maintaining epistemic inclusivity in AI safety research is not only an intellectual necessity but also a crucial step in preparing alignment efforts for the challenges posed by rapid AI advancements. (shrink)

We argue that the later Wittgenstein’s philosophy of language and mathematics, substantially focused on rule-following, is relevant to understand and improve on the Artificial Intelligence (AI) alignment problem: his discussions on the categories that influence alignment between humans can inform about the categories that should be controlled to improve on the alignment problem when creating large data sets to be used by supervised and unsupervised learning algorithms, as well as when introducing hard coded guardrails for AI models. (...) We cast these considerations in a model of human–human and human–machine alignment and sketch basic alignment strategies based on these categories and further reflections on rule-following like the notion of meaning as use. To sustain the validity of these considerations, we also show that successful techniques employed by AI safety researchers to better align new AI systems with our human goals are congruent with the stipulations that we derive from the later Wittgenstein’s philosophy. However, their application may benefit from the added specificities and stipulations of our framework: it extends on the current efforts and provides further, specific AI alignment techniques. Thus, we argue that the categories of the model and the core alignment strategies presented in this work can inform further AI alignment techniques. (shrink)

Tae Wan Kim, John Hooker, and Thomas Donaldson make an attempt, in recent articles, to solve the alignment problem. As they define the alignment problem, it is the issue of how to give AI systems moral intelligence. They contend that one might program machines with a version of Kantian ethics cast in deontic modal logic. On their view, machines can be aligned with human values if such machines obey principles of universalization and autonomy, as well as a deontic (...) utilitarian principle. Programming machines to do so might be useful, in their view, for applications such as future autonomous vehicles. Their proposal draws both on traditional logic-based and contemporary connectionist approaches, to fuse factual information with normative principles. I will argue that this approach makes demands of machines that go beyond what is currently feasible, and may extend past the limits of the possible for AI. I also argue that a deontological ethics for machines should place greater stress on the formula of humanity of the Kantian categorical imperative. On this principle, one ought never treat a person as a mere means. Recognition of what makes a person a person requires ethical insight. Similar insight is needed to tell treatment as a means from treatment as a mere means. The resources in Kim, Hooker, and Donaldson’s approach is insufficient for this reason. Hesitation regarding deployment of autonomous machines is warranted in light of these alignment concerns. (shrink)

Foundation models such as GPT-4 are fine-tuned to avoid unsafe or otherwise problematic behavior, such as helping to commit crimes or producing racist text. One approach to fine-tuning, called reinforcement learning from human feedback, learns from humans' expressed preferences over multiple outputs. Another approach is constitutional AI, in which the input from humans is a list of high-level principles. But how do we deal with potentially diverging input from humans? How can we aggregate the input into consistent data about "collective" (...) preferences or otherwise use it to make collective choices about model behavior? In this paper, we argue that the field of social choice is well positioned to address these questions, and we discuss ways forward for this agenda, drawing on discussions in a recent workshop on Social Choice for AI Ethics and Safety held in Berkeley, CA, USA in December 2023. (shrink)

The dominant practice of AI alignment assumes (1) that preferences are an adequate representation of human values, (2) that human rationality can be understood in terms of maximizing the satisfaction of preferences, and (3) that AI systems should be aligned with the preferences of one or more humans to ensure that they behave safely and in accordance with our values. Whether implicitly followed or explicitly endorsed, these commitments constitute what we term apreferentistapproach to AI alignment. In this paper, (...) we characterize and challenge the preferentist approach, describing conceptual and technical alternatives that are ripe for further research. We first survey the limits of rational choice theory as a descriptive model, explaining how preferences fail to capture the thick semantic content of human values, and how utility representations neglect the possible incommensurability of those values. We then critique the normativity of expected utility theory (EUT) for humans and AI, drawing upon arguments showing how rational agents need not comply with EUT, while highlighting how EUT is silent on which preferences are normatively acceptable. Finally, we argue that these limitations motivate a reframing of the targets of AI alignment: Instead of alignment with the preferences of a human user, developer, or humanity-writ-large, AI systems should be aligned with normative standards appropriate to their social roles, such as the role of a general-purpose assistant. Furthermore, these standards should be negotiated and agreed upon by all relevant stakeholders. On this alternative conception of alignment, a multiplicity of AI systems will be able to serve diverse ends, aligned with normative standards that promote mutual benefit and limit harm despite our plural and divergent values. (shrink)

The problem of AI alignment has parallels in Kantian ethics and can benefit from its concepts and arguments. The Kantian framework allows us to better answer the question of what exactly AI is being aligned to, what are the problems of alignment of rational agents in general, and what are the prospects for achieving a state of alignment. Having described the state of discussions about alignment in AI, I will reformulate them in Kantian terms. Thus, the (...) process of alignment is captured by the concept of enlightenment, and for the final state of alignment in Kant’s lexicon there is the concept of the “kingdom of ends.” I will argue that the discourse of alignment and the Kantian ethical program 1) are devoted to the same general end of harmonizing the thinking and acting of rational agents, 2) encounter similar difficulties, well known in the Kantian discussions with its comparatively longer history, and 3) for a number of reasons lying on the side of humanity, do not have and, despite the hopes and attitudes of some participants in the AI discussions, will not have a theoretically rigorous, harmonious and practically implementable, conflict-free solution – alignment will remain a regulative idea in the Kantian sense, but will not become a reality. (shrink)

The Alignment Problem in artificial intelligence concerns how to insure that artificial general intelligence (AGI) conforms to human goals and values and remains under human control. The concept of general intelligence, modelled on human and animal behavior, lacks coherence. The ideal of autonomy inherent in AGI conflicts with the ideal of external control. Truly autonomous agents are necessarily _embodied,_ but embodiment implies more than physical instantiation or sensory input. It means being an autopoietic system (like a natural organism), with (...) its own priorities and values, which may compete and conflict with those of humans. Ambiguous terms and concepts, and problematic notions such as ‘orthogonality’, are critically examined, as well as inconsistencies in thought and expectation concerning AGI. Two paradigmatic approaches to the Alignment Problem are compared: Stuart Russell’s and Eric Drexler’s. Large Language Models are discussed in the context of the Turing Test. It is concluded that task-oriented tools, not autonomous agents, should be the goal of AI research. (shrink)

The threat from possible future superintelligent AI has given rise to discussion of the so-called “value alignment problem”. This is the problem of how to ensure artificially intelligent systems align with human values, and thus (hopefully) mitigate risks associated with them. Naturally, AI value alignment is often discussed in relation to morally relevant values, such as the value of human lives or human wellbeing. However, solutions to the value alignment problem target all human values, not only morally (...) relevant ones. Is there a value alignment problem in other domains? In this paper, I explore whether the AI value alignment problem extends beyond morally relevant values to include aesthetic values. I demonstrate that the value alignment problem as typically framed includes aesthetic values, and, using examples from computer vision, put forward that AI may be misaligned with human values in the artistic realm. Whilst misalignment may be cause for concern when considering AI creativity, I argue that aesthetic value is a case in which we do not want AI to be fully aligned with human values. In doing so, I offer support to Peterson’s moderate value alignment thesis. (shrink)

When discussing AI alignment, we usually refer to the problem of teaching or training advanced autonomous AI systems to make decisions that are aligned with human values or preferences. Proponents of this approach believe it can be employed as means to stay in control over sophisticated intelligent systems, thus avoiding certain existential risks. We identify three general obstacles on the path to implementation of value alignment: a technological/technical obstacle, a normative obstacle, and a calibration problem. Presupposing, for the (...) purposes of this discussion, that the technical and normative problems are solved, we focus on the problem of how to calibrate a system, for a specific value, to be on a specific location within a spectrum stretching between righteous and normal or average human behavior. Calibration, or more specifically mis-calibration, also raises the issue of trustworthiness. If we cannot trust AI systems to perform tasks the way we intended, we would not use them on our roads and at our homes. In an era where we strive to construct autonomous machines endowed with common sense, reasoning abilities and a connection to the world, so they would be able to act in alignment with human values, such mis-calibrations can make the difference between trustworthy and untrustworthy systems. (shrink)

In lieu of an abstract, here is a brief excerpt of the content:Toleration and Justice in the Laozi:Engaging with Tao Jiang's Origins of Moral-Political Philosophy in Early ChinaAi Yuan (bio)IntroductionThis review article engages with Tao Jiang's ground-breaking monograph on the Origins of Moral-Political Philosophy in Early China with particular focus on the articulation of toleration and justice in the Laozi (otherwise called the Daodejing).1 Jiang discusses a naturalistic turn and the re-alignment of values in the Laozi, resulting in a (...) naturalization of justice (impartiality) by rejecting artificial humaneness and rigid hierarchical moral-political structures. In this Laozian just world, there is no room for human intervention and rigid top-down enforcement of values, thus leaving justice to naturalized Heaven or Dao as the ultimate source of the cosmos.Based on Jiang's interpretative context, I show that there is a twofold justification for the "paradoxical" and "elusive"2 value of political toleration.3 A negative expression of toleration focuses on non-interference with choices, and non-enforcement of values toward those whom one reasonably disagrees with and regards as morally wrong. Such an attitude of toleration results in practical advantages such as avoiding entrenchment in bloodshed, abandonment, and deprivation.A positive expression of toleration operating in the Laozian ideal world requires equal protection of people as capable knowers, including those to whom we object. Such protection includes their ways of expression and an unbiased recognition of them as equally capable knowers so that they are heard without bias and prejudice. Such a kind of toleration is revealed as a constituent component of justice since it enables equal contribution toward the shaping of society without discrimination. In other words, toleration is seen as naturalized justice in a Laozian world within which no one is "being wronged with the capacity as a knower"—a kind of epistemic justice articulated by Miranda Fricker (Fricker 2007).4 [End Page 466]This article discusses how Jiang's political reconstruction of Laozian philosophy contains the seeds for a discussion of toleration, a value associated with distributive justice and against epistemic injustice. First, it introduces Jiang's arguments on a naturalization of justice and impartiality as Heavenly attributes in the Laozi. Second, it articulates the value of tolerance in the Laozi. Finally, I compare Laozian toleration with Confucian toleration using youwei- wuwei metaethical criticism.Moral-Political Philosophy in the LaoziJiang's interpretative framework about the origins of distributive justice in the Laozi starts with a discussion of the cosmogonic-mystical worldviews that signal "a new understanding of the nature of the cosmos, as well as a broad reorientation in the Heaven-human relationship in the mid-Warring States period" (p. 192). Differing from the interests in the origins of human culture or civilization in the Ru and Mo traditions, Jiang agrees with Franklin Perkins and identifies a "cosmogonic turn" in the late fourth century b.c. that reveals a demotion of anthropomorphic Heaven. By paring Heaven with earth and accordingly decentering human beings in the intellectual discourse, the Laozi directs equal values to the ten thousand things, with humans as just one among them (Perkins 2016, quoted in Jiang 2021, p. 196).5 Laozi's philosophy thus stands out by rejecting Heaven as caring for human affairs while taking Dao as the ultimate origin of the cosmos, as claimed in chapter 5 of the received text: "Heaven and earth are not humane; they treat ten-thousand things in the world as straw dogs; the sage is not humane; he treats people as straw dogs" (p. 199).With Jiang's reading of a naturalistic heaven, we see a re-alignment between cosmos and the human since sage-rulers follow the Heaven-Dao. The Laozian Dao is "self-so-ing" (ziran 自然), "accommodating" (rong 容), and "impartial" (gong 公), and thus marks a sharp contrast to Confucian familial bias resulting in "actively intervening in human affairs on behalf of the (certain) humans by appointing them to carry out its mission" (p. 200). With the sage emulating the ultimate Dao, the Laozi replaces the Confucian anthropocentric Heaven, which focuses on partiality (qin 親) and humaneness (ren 仁).This Laozian understanding of the cosmos also grounds its philosophical departure from Mohism despite both having a political... (shrink)

The Salivaomics Knowledge Base (SKB) is designed to serve as a computational infrastructure that can permit global exploration and utilization of data and information relevant to salivaomics. SKB is created by aligning (1) the saliva biomarker discovery and validation resources at UCLA with (2) the ontology resources developed by the OBO (Open Biomedical Ontologies) Foundry, including a new Saliva Ontology (SALO). We define the Saliva Ontology (SALO; http://www.skb.ucla.edu/SALO/) as a consensus-based controlled vocabulary of terms and relations dedicated to the salivaomics (...) domain and to saliva-related diagnostics following the principles of the OBO (Open Biomedical Ontologies) Foundry. The Saliva Ontology is an ongoing exploratory initiative. The ontology will be used to facilitate salivaomics data retrieval and integration across multiple fields of research together with data analysis and data mining. The ontology will be tested through its ability to serve the annotation ('tagging') of a representative corpus of salivaomics research literature that is to be incorporated into the SKB. Background Saliva (oral fluid) is an emerging biofluid for non-invasive diagnostics used in the detection of human diseases. The need to advance saliva research is strongly emphasized by the National Institute of Dental and Craniofacial Research (NIDCR), and is included in the NIDCR's 2004- 2009 expert panel long-term research agenda [1]. The ability to monitor health status, disease onset, progression, recurrence and treatment outcome through noninvasive means is highly important to advancing health care management. Saliva is a perfect medium to be explored for personalized individual medicine including diagnostics, offering a non-invasive, easy to obtain means for detecting and monitoring diseases. Saliva testing potentially allows the patient to collect their own saliva samples at home, yielding convenience for the patient and savings in health costs, and facilitating multiple sampling. Specimen collection is less objectionable to patients and easier in children and elderly individuals. Due to these advantages. (shrink)

The challenge of AI alignment is not just a technological issue but fundamentally an epistemic one. AI safety research predominantly relies on empirical validation, often detecting failures only after they manifest. However, certain risks—such as deceptive alignment and goal misspecification—may not be empirically testable until it is too late, necessitating a shift toward leading-indicator logical reasoning. This paper explores how mainstream AI research systematically filters out deep epistemic insight, hindering progress in AI safety. We assess the rarity of (...) such insights, conduct an experiment testing large language models for epistemic blind spots, and propose structural reforms, including contrarian epistemic screening, decentralized collective intelligence mechanisms, and epistemic challenge platforms. Our findings suggest that while AGI may emerge through incremental engineering, ensuring its safe alignment likely requires an epistemic paradigm shift. (shrink)

Member States of the United Nations proposed Seventeen Sustainable Development Goals (SDGs) in 2015, emphasizing the well-being of people, planet, prosperity, peace, and partnership. Countries are expected to work diligently to achieve these goals by the year 2030. The paths chosen to achieve the SDGs depend on each country’s specific needs, challenges, and opportunities. This contribution conducts a bibliometric study of selected SDG research related to hunger and climate change among countries of the North Asia Pacific region. A review of (...) literature related to the seventeen SDGs is provided to update the current research in SDGs as a background for the regional bibliometric search. The academic published literature related to this work were collected and text mined to create a research database for trend analysis. In this research, the natural language model KeyBERT and a text mining algorithm for normalized term frequency were used to extract key terminologies for bibliometric analysis. The study provides an overview of the seventeen SDGs in the literature review and then focuses on SDG 2 (zero hunger) and SDG 13 (climate action) publication trends. These two SDGs are in close alignment with sustainable food systems and agricultural development. The conclusion indicates that greater cooperation (SDG 17) is critical to improving the quality of life within the region and in building an ethical framework that restricts external exploitation of resources that benefit newly developed economies over developing economies. (shrink)

Value alignment is essential for ensuring that AI systems act in ways that are consistent with human values. Existing approaches, such as reinforcement learning with human feedback and constitutional AI, however, exhibit power asymmetries and lack transparency. These “authoritarian” approaches fail to adequately accommodate a broad array of human opinions, raising concerns about whose values are being prioritized. In response, we introduce the Dynamic Value Alignment approach, theoretically grounded in the principles of parallel constraint satisfaction, which models moral (...) reasoning as a dynamic process that balances multiple value principles. Our approach also enhances users’ moral and epistemic agency by granting users greater control over the values that influence AI behavior. As a more user-centric, transparent, and participatory framework for AI ethics, our approach not only addresses the democratic deficits inherent in current practices but also ensures that AI systems are flexibly aligned with a diverse array of human values. (shrink)

This paper offers three contributions to the burgeoning movements of AI for Social Good (AI4SG) and AI and the United Nations Sustainable Development Goals (SDGs). First, we introduce the SDG-Intense Evaluation framework (SDGIE) that aims to situate variegated automated/AI models in a larger ecosystem of computational approaches to advance the SDGs. To foster knowledge collaboration for solving complex social and environmental problems encompassed by the SDGs, the SDGIE framework details a benchmark structure of data-algorithm-output to effectively standardize AI approaches to (...) the SDGs. Second, as a specific instantiation of the SDGIE framework, the SDG Impact Intensity Model (SDGIIM) is theoretically and operationally established. SDGIIM embeds expert decision-making and SDG keyword banks in textual data processing to determine overall SDG “impact intensity.” Ideally, SDGIIM can be applied to textual data sets from any sector or discipline: academia, business, government, non-profit, civil society, etc. Third, the SDGIIM instantiation is applied to the specific domain of academic journal rating systems as a case study. Traditionally, academic journals have been evaluated on loosely conceived and empirically shaky notions of ‘quality.’ Aligned with the trend of AI4SG and broader calls to action, ‘impact’ is rapidly becoming the primary normative consideration for assessing academic journals. We hypothesize and demonstrate that SDGIIM is capable of producing evaluations aligned with experts’ expectations of SDG impact intensity; the consistent analysis and rating of textual data sets that embody the SDGs with varying degrees of meaning and, ultimately, promote positive impact on the actual material conditions of the world. (shrink)

How can one build AI systems such that they pursue the goals their designers want them to pursue? This is the alignment problem. Numerous authors have raised concerns that, as research advances and systems become more powerful over time, misalignment might lead to catastrophic outcomes, perhaps even to the extinction or permanent disempowerment of humanity. In this paper, I analyze the severity of this risk based on current instances of misalignment. More specifically, I argue that contemporary large language models (...) and game-playing agents are sometimes misaligned. These cases suggest that misalignment tends to have a variety of features: misalignment can be hard to detect, predict and remedy, it does not depend on a specific architecture or training paradigm, it tends to diminish a system’s usefulness and it is the default outcome of creating AI via machine learning. Subsequently, based on these features, I show that the risk of AI alignment magnifies with respect to more capable systems. Not only might more capable systems cause more harm when misaligned, aligning them should be expected to be more difficult than aligning current AI. (shrink)

Some researchers have recognized that privileged communities dominate the discourse on AI Ethics, and other voices need to be heard. As such, we identify the current ethics milieu as arising from WEIRD (Western, Educated, Industrialized, Rich, Democratic) contexts, and aim to expand the discussion to non-WEIRD global communities, who are also stakeholders in global sociotechnical systems. We argue that accounting for honor, along with its values and related concepts, would better approximate a global ethical perspective. This complex concept already underlies (...) some of the WEIRD discourse on AI ethics, but certain cultural forms of honor also bring overlooked issues and perspectives to light. We first describe honor according to recent empirical and philosophical scholarship. We then review “consensus” principles for AI ethics framed from an honor-based perspective, grounding comparisons and contrasts via example settings such as content moderation, job hiring, and genomics databases. A better appreciation of the marginalized concept of honor could, we hope, lead to more productive AI value alignment discussions, and to AI systems that better reflect the needs and values of users around the globe. (shrink)

Providing meaningful and actionable explanations for end-users is a situated problem requiring the intersection of multiple disciplines to address social, operational, and technical challenges. However, the explainable artificial intelligence community has not commonly adopted or created tangible design tools that allow interdisciplinary work to develop reliable AI-powered solutions. This paper proposes a formative architecture that defines the explanation space from a user-inspired perspective. The architecture comprises five intertwined components to outline explanation requirements for a task: (1) the end-users’ mental models, (...) (2) the end-users’ cognitive process, (3) the user interface, (4) the Human-Explainer Agent, and (5) the agent process. We first define each component of the architecture. Then, we present the Abstracted Explanation Space, a modeling tool that aggregates the architecture’s components to support designers in systematically aligning explanations with end-users’ work practices, needs, and goals. It guides the specifications of what needs to be explained (content: end-users’ mental model), why this explanation is necessary (context: end-users’ cognitive process), to delimit how to explain it (format: Human-Explainer Agent and user interface), and when the explanations should be given. We then exemplify the tool’s use in an ongoing case study in the aircraft maintenance domain. Finally, we discuss possible contributions of the tool, known limitations or areas for improvement, and future work to be done. (shrink)

Upon previous research, the matrilineal system seems to oppose Islamic teaching. However, the matrilineal system practiced by the Minangkabau society in West Sumatra, Indonesia has its uniqueness. Thus, this study aims to examine the correlation between the Qur’an and gender roles within the context of Minangkabau customs, specifically focusing on the matrilineal aspect. The present study employs qualitative methods for conducting library research through critical analysis. This study discovered that the matrilineal system practiced by the Minangkabau society aligns with Qur’anic (...) teachings that respect the position of women in Islam. In the matrilineal system of Minangkabau, women hold significant strength and play an influential role in cultural and traditional practices. The Minangkabau tribe’s matrilineal system encompasses three key elements: marriage, inheritance and offspring. The marriage system is exogamous. Additionally, the practice of marriage is localised, with the husband typically residing in the wife’s household. Their inheritance is divided into pusako tinggi [from ancestors from generation to generation] and pusako randah (from the efforts of married couple). Women get pusako tinggi as they serve as protectors and a symbol of their community. In comparison, men take the responsibility of overseeing its preservation to prevent any form of misuse. Pusako randah is distributed as Islamic teaching. Lineage determination is primarily based on paternal lineage, aligning with Islamic teaching. Upon comprehensive analysis, it becomes evident that there is no conflict between the matrilineal system and Islamic teachings, particularly concerning principles of gender equality. Both women and men have significant roles, as the Qur’an provides extensive explanations regarding gender equality.Contribution: This research holds significant findings in establishing a great relation between Islamic teachings based on the Qur’an and the matrilineal principles practiced by the Minangkabau tribe. (shrink)

This paper categorizes current approaches to AI ethics into four perspectives and briefly summarizes them: (1) Case studies and technical trend surveys, (2) AI governance, (3) Technologies for AI alignment, (4) Philosophy. In the second half, we focus on the fourth perspective, the philosophical approach, within the context of applied ethics. In particular, the explicability of AI may be an area in which scientists, engineers, and AI developers are expected to engage more actively relative to other ethical issues in (...) AI. (shrink)

Since the release of ChatGPT, there has been a lot of debate about whether AI systems pose an existential risk to humanity. This paper develops a general framework for thinking about the existential risk of AI systems. We analyze a two-premise argument that AI systems pose a threat to humanity. Premise one: AI systems will become extremely powerful. Premise two: if AI systems become extremely powerful, they will destroy humanity. We use these two premises to construct a taxonomy of ‘survival (...) stories’, in which humanity survives into the far future. In each survival story, one of the two premises fails. Either scientific barriers prevent AI systems from becoming extremely powerful; or humanity bans research into AI systems, thereby preventing them from becoming extremely powerful; or extremely powerful AI systems do not destroy humanity, because their goals prevent them from doing so; or extremely powerful AI systems do not destroy humanity, because we can reliably detect and disable systems that have the goal of doing so. We argue that different survival stories face different challenges. We also argue that different survival stories motivate different responses to the threats from AI. Finally, we use our taxonomy to produce rough estimates of ‘P(doom)’, the probability that humanity will be destroyed by AI. (shrink)

As artificial intelligence (AI) continues to advance, one key challenge is ensuring that AI aligns with certain values. However, in the current diverse and democratic society, reaching a normative consensus is complex. This paper delves into the methodological aspect of how AI ethicists can effectively determine which values AI should uphold. After reviewing the most influential methodologies, we detail an intuitionist research agenda that offers guidelines for aligning AI applications with a limited set of reliable moral intuitions, each underlying a (...) refined cooperative view of AI. We discuss appropriate epistemic tools for collecting, filtering, and justifying moral intuitions with the aim of reducing cognitive and social biases. The proposed methodology facilitates a large collective participation in AI alignment, while ensuring the reliability of the considered moral judgments. (shrink)

We outline a four-step process for ML/AI developers to align development choices with multiple values, by adapting a widely-utilized framework from bioethics: (1) identify the values that matter, (2) specify identified values, (3) find solution spaces that allow for maximal alignment with identified values, and 4) make hard choices if there are unresolvable trade-offs between the identified values. Key to this approach is identifying resolvable trade-offs between values (Step 3). We survey ML/AI methods that could be used to this (...) end, identifying approaches at each stage of the development process. All steps should be guided by community engagement. The framework outlines what it means to build a value-aligned ML/AI system, providing development teams with practical guidance to maximize the chances their work has desirable impacts. (shrink)

Healthcare systems are grappling with critical challenges, including chronic diseases in aging populations, unprecedented health care staffing shortages and turnover, scarce resources, unprecedented demands and wait times, escalating healthcare expenditure, and declining health outcomes. As a result, policymakers and healthcare executives are investing in artificial intelligence (AI) solutions to increase operational efficiency, lower health care costs, and improve patient care. However, current level of investment in developing healthcare AI among members of the global digital health partnership does not seem to (...) yield a high return yet. This is mainly due to underinvestment in the supporting infrastructure necessary to enable the successful implementation of AI. If a healthcare-specific AI winter is to be avoided, it is paramount that this disparity in the level of investment in the development of AI itself and in the development of the necessary supporting system components is evened out. (shrink)

Artificial Intelligence (AI) must be directed at humane ends. The development of AI has produced great uncertainties of ensuring AI alignment with human values (AI value alignment) through AI operations from design to use. For the purposes of addressing this problem, we adopt the phenomenological theories of material values and technological mediation to be that beginning step. In this paper, we first discuss the AI value alignment from the relevant AI studies. Second, we briefly present what are (...) material values and technological mediation and reflect on the AI value alignment through the lenses of these theories. We conclude that a set of finite human values can be defined and adapted to the stable life tasks that AI systems will be called upon to accomplish. The AI value alignment can also be fostered between designers and users through technological mediation. Upon that foundation, we propose a set of common principles to understand the AI value alignment through phenomenological theories. This paper contributes the unique knowledge of phenomenological theories to the discourse on AI alignment with human values. (shrink)

Inchul Yum (2024) argues that the widespread adoption of language agent architectures would likely increase the risk posed by AI by simplifying the process of aligning artificial systems with human values and thereby making it easier for malicious actors to use them to cause a variety of harms. Yum takes this to be an example of a broader phenomenon: progress on the alignment problem is likely to be net safety-negative because it makes artificial systems easier for malicious actors to (...) control. I offer some reasons for skepticism about this surprising and pessimistic conclusion. (shrink)

This paper explores the potential of integrating ancient educational principles from diverse eastern cultures into modern AI ethics curricula. It draws on the rich educational traditions of ancient China, India, Arabia, Persia, Japan, Tibet, Mongolia, and Korea, highlighting their emphasis on philosophy, ethics, holistic development, and critical thinking. By examining these historical educational systems, the paper establishes a correlation with modern AI ethics principles, advocating for the inclusion of these ancient teachings in current AI development and education. The proposed integration (...) aims to provide a comprehensive education that not only encompasses foundational knowledge but also advanced learning, thereby equipping future AI professionals with the necessary tools to develop AI systems that are ethically responsible, culturally aware, and aligned with human values such as fairness, safety, transparency, and collaboration. This approach not only addresses the AI alignment problem but also fosters cultural harmony and global understanding, which are crucial in an increasingly interconnected world. The paper posits that the wisdom of ancient educational systems, when harmonized with modern AI ethics, can guide the development of AI technologies that are beneficial for humanity, ensuring these advancements are not just technologically sound but also ethically and culturally informed. (shrink)

This paper looks at philosophical questions that arise in the context of AI alignment. It defends three propositions. First, normative and technical aspects of the AI alignment problem are interrelated, creating space for productive engagement between people working in both domains. Second, it is important to be clear about the goal of alignment. There are significant differences between AI that aligns with instructions, intentions, revealed preferences, ideal preferences, interests and values. A principle-based approach to AI alignment, (...) which combines these elements in a systematic way, has considerable advantages in this context. Third, the central challenge for theorists is not to identify ‘true’ moral principles for AI; rather, it is to identify fair principles for alignment that receive reflective endorsement despite widespread variation in people’s moral beliefs. The final part of the paper explores three ways in which fair principles for AI alignment could potentially be identified. (shrink)

This paper uses famous problems from philosophy of science and philosophical psychology—underdetermination of theory by evidence, Nelson Goodman’s new riddle of induction, theory-ladenness of observation, and “Kripkenstein’s” rule-following paradox—to show that it is empirically impossible to reliably interpret which functions a large language model (LLM) AI has learned, and thus, that reliably aligning LLM behavior with human values is provably impossible. Sections 2 and 3 show that because of how complex LLMs are, researchers must interpret their learned functions largely in (...) terms of empirical observations of their outputs and network behavior. Sections 4–7 then show that for every “aligned” function that might appear to be confirmed by empirical observation, there is always an infinitely larger number of “misaligned”, arbitrarily time-limited functions equally consistent with the same data. Section 8 shows that, from an empirical perspective, we can thus never reliably infer that an LLM or subcomponent of one has learned any particular function at all before any of an uncountably large number of unpredictable future conditions obtain. Finally, Section 9 concludes that the probability of LLM “misalignment” is—at every point in time, given any arbitrarily large body of empirical evidence—always vastly greater than the probability of “alignment.”. (shrink)

This paper considers challenges resulting from the use of advanced artificial judicial intelligence (AAJI). We argue that these challenges should be considered through the lens of value alignment. Instead of discussing why specific goals and values, such as fairness and nondiscrimination, ought to be implemented, we consider the question of how AAJI can be aligned with goals and values more generally, in order to be reliably integrated into legal and judicial systems. This value alignment framing draws on AI (...) safety and alignment literature to introduce two otherwise neglected considerations for AAJI safety: specification and assurance. We outline diverse research directions and suggest the adoption of assurance and specification mechanisms as the use of AI in the judiciary progresses. While we focus on specification and assurance to illustrate the value of the AI safety and alignment literature, we encourage researchers in law and philosophy to consider what other lessons may be drawn. (shrink)

What makes safety claims about general purpose AI systems such as large language models trustworthy? We show that rather than the capabilities of security tools such as alignment and red teaming procedures, it is security practices based on these tools that contributed to reconfiguring the image of AI safety and made the claims acceptable. After showing what causes the gap between the capabilities of security tools and the desired safety guarantees, we critically investigate how AI security practices attempt to (...) fill the gap and identify several shortcomings in diversity and participation. We found that these security practices are part of securitization processes aiming to support (commercial) development of general purpose AI systems whose trustworthiness can only be imperfectly tested instead of guaranteed. We conclude by offering several improvements to the current AI security practices. (shrink)

Many researchers and intellectuals warn about extreme risks from artificial intelligence. However, these warnings typically came without systematic arguments in support. This paper provides an argument that AI will lead to the permanent disempowerment of humanity, e.g. human extinction, by 2100. It rests on four substantive premises which it motivates and defends: first, the speed of advances in AI capability, as well as the capability level current systems have already reached, suggest that it is practically possible to build AI systems (...) capable of disempowering humanity by 2100. Second, due to incentives and coordination problems, if it is possible to build such AI, it will be built. Third, since it appears to be a hard technical problem to build AI which is aligned with the goals of its designers, and many actors might build powerful AI, misaligned powerful AI will be built. Fourth, because disempowering humanity is useful for a large range of misaligned goals, such AI will try to disempower humanity. If AI is capable of disempowering humanity and tries to disempower humanity by 2100, then humanity will be disempowered by 2100. This conclusion has immense moral and prudential significance. (shrink)

This paper explores the development of an ethical guardrail framework for AI systems, emphasizing the importance of customizable guardrails that align with diverse user values and underlying ethics. We address the challenges of AI ethics by proposing a structure that integrates rules, policies, and AI assistants to ensure responsible AI behavior, while comparing the proposed framework to the existing state-of-the-art guardrails. By focusing on practical mechanisms for implementing ethical standards, we aim to enhance transparency, user autonomy, and continuous improvement in (...) AI systems. Our approach accommodates ethical pluralism, offering a flexible and adaptable solution for the evolving landscape of AI governance. The paper concludes with strategies for resolving conflicts between ethical directives, underscoring the present and future need for robust, nuanced and context-aware AI systems. (shrink)

This paper explores the interplay between AI metrics and policymaking by examining the conceptual and methodological frameworks of global AI metrics and their alignment with National Artificial Intelligence Strategies (NAIS). Through topic modeling and qualitative content analysis, key thematic areas in NAIS are identified. The findings suggest a misalignment between the technical and economic focus of global AI metrics and the broader societal and ethical priorities emphasized in NAIS. This highlights the need to recalibrate AI evaluation frameworks to include (...) ethical and other social considerations, aligning AI advancements with the United Nations Sustainable Development Goals (SDGs) for an inclusive, ethical, and sustainable future. (shrink)

Critics currently argue that applied ethics approaches to artificial intelligence (AI) are too principles-oriented and entail a theory–practice gap. Several applied ethical approaches try to prevent such a gap by conceptually translating ethical theory into practice. In this article, we explore how the currently most prominent approaches of AI ethics translate ethics into practice. Therefore, we examine three approaches to applied AI ethics: the embedded ethics approach, the ethically aligned approach, and the Value Sensitive Design (VSD) approach. We analyze each (...) of these three approaches by asking how they understand and conceptualize theory and practice. We outline the conceptual strengths as well as their shortcomings: an embedded ethics approach is context-oriented but risks being biased by it; ethically aligned approaches are principles-oriented but lack justification theories to deal with trade-offs between competing principles; and the interdisciplinary Value Sensitive Design approach is based on stakeholder values but needs linkage to political, legal, or social governance aspects. Against this background, we develop a meta-framework for applied AI ethics conceptions with three dimensions. Based on critical theory, we suggest these dimensions as starting points to critically reflect on the conceptualization of theory and practice. We claim, first, that the inclusion of the dimension of affects and emotions in the ethical decision-making process stimulates reflections on vulnerabilities, experiences of disregard, and marginalization already within the AI development process. Second, we derive from our analysis that considering the dimension of justifying normative background theories provides both standards and criteria as well as guidance for prioritizing or evaluating competing principles in cases of conflict. Third, we argue that reflecting the governance dimension in ethical decision-making is an important factor to reveal power structures as well as to realize ethical AI and its application because this dimension seeks to combine social, legal, technical, and political concerns. This meta-framework can thus serve as a reflective tool for understanding, mapping, and assessing the theory–practice conceptualizations within AI ethics approaches to address and overcome their blind spots. (shrink)

As the capabilities of artificial intelligence systems improve, it becomes important to constrain their actions to ensure their behaviour remains beneficial to humanity. A variety of ethical, legal and safety-based frameworks have been proposed as a basis for designing these constraints. Despite their variations, these frameworks share the common characteristic that decision-making must consider multiple potentially conflicting factors. We demonstrate that these alignment frameworks can be represented as utility functions, but that the widely used Maximum Expected Utility paradigm provides (...) insufficient support for such multiobjective decision-making. We show that a Multiobjective Maximum Expected Utility paradigm based on the combination of vector utilities and non-linear action–selection can overcome many of the issues which limit MEU’s effectiveness in implementing aligned AI. We examine existing approaches to multiobjective AI, and identify how these can contribute to the development of human-aligned intelligent agents. (shrink)

We apply Ken Wilber's Integral Theory to AI governance, demonstrating its ability to systematize diverse approaches in the current multifaceted AI governance landscape. By analyzing ethical considerations, technological standards, cultural narratives, and regulatory frameworks through Integral Theory's four quadrants, we offer a comprehensive perspective on governance needs. This approach aligns AI governance with human values, psychological well-being, cultural norms, and robust regulatory standards. Integral Theory’s emphasis on interconnected individual and collective experiences addresses the deeper aspects of AI-related issues. Additionally, we (...) propose using Integral Theory as a methodology for literature reviews to overcome the fragmented understanding often seen in traditional reviews of AI governance. (shrink)

This paper explores the important role of critical science, and in particular of post-colonial and decolonial theories, in understanding and shaping the ongoing advances in artificial intelligence. Artificial intelligence is viewed as amongst the technological advances that will reshape modern societies and their relations. While the design and deployment of systems that continually adapt holds the promise of far-reaching positive change, they simultaneously pose significant risks, especially to already vulnerable peoples. Values and power are central to this discussion. Decolonial theories (...) use historical hindsight to explain patterns of power that shape our intellectual, political, economic, and social world. By embedding a decolonial critical approach within its technical practice, AI communities can develop foresight and tactics that can better align research and technology development with established ethical principles, centring vulnerable peoples who continue to bear the brunt of negative impacts of innovation and scientific progress. We highlight problematic applications that are instances of coloniality, and using a decolonial lens, submit three tactics that can form a decolonial field of artificial intelligence: creating a critical technical practice of AI, seeking reverse tutelage and reverse pedagogies, and the renewal of affective and political communities. The years ahead will usher in a wave of new scientific breakthroughs and technologies driven by AI research, making it incumbent upon AI communities to strengthen the social contract through ethical foresight and the multiplicity of intellectual perspectives available to us, ultimately supporting future technologies that enable greater well-being, with the goal of beneficence and justice for all. (shrink)

The call to make artificial intelligence (AI) more democratic, or to “democratize AI,” is sometimes framed as a promising response for mitigating algorithmic injustice or making AI more aligned with social justice. However, the notion of “democratizing AI” is elusive, as the phrase has been associated with multiple meanings and practices, and the extent to which it may help mitigate algorithmic injustice is still underexplored. In this paper, based on a socio-technical understanding of algorithmic injustice, I examine three notable notions (...) of democratizing AI and their associated measures—democratizing AI use, democratizing AI development, and democratizing AI governance—regarding their respective prospects and limits in response to algorithmic injustice. My examinations reveal that while some versions of democratizing AI bear the prospect of mitigating the concern of algorithmic injustice, others are somewhat limited and might even function to perpetuate unjust power hierarchies. This analysis thus urges a more fine-grained discussion on how to democratize AI and suggests that closer scrutiny of the power dynamics embedded in the socio-technical structure can help guide such explorations. (shrink)

The value alignment problem for artificial intelligence (AI) asks how we can ensure that the “values”—i.e., objective functions—of artificial systems are aligned with the values of humanity. In this paper, I argue that linguistic communication is a necessary condition for robust value alignment. I discuss the consequences that the truth of this claim would have for research programmes that attempt to ensure value alignment for AI systems—or, more loftily, those programmes that seek to design robustly beneficial or (...) ethical artificial agents. (shrink)

AI systems that demonstrate significant bias or lower than claimed accuracy, and resulting in individual and societal harms, continue to be reported. Such reports beg the question as to why such systems continue to be funded, developed and deployed despite the many published ethical AI principles. This paper focusses on the funding processes for AI research grants which we have identified as a gap in the current range of ethical AI solutions such as AI procurement guidelines, AI impact assessments and (...) AI audit frameworks. We highlight the responsibilities of funding bodies to ensure investment is channelled towards trustworthy and safe AI systems and provides case studies as to how other ethical funding principles are managed. We offer a first sight of two proposals for funding bodies to consider regarding procedures they can employ. The first proposal is for the inclusion of a Trustworthy AI Statement’ section in the grant application form and offers an example of the associated guidance. The second proposal outlines the wider management requirements of a funding body for the ethical review and monitoring of funded projects to ensure adherence to the proposed ethical strategies in the applicants Trustworthy AI Statement. The anticipated outcome for such proposals being employed would be to create a ‘stop and think’ section during the project planning and application procedure requiring applicants to implement the methods for the ethically aligned design of AI. In essence it asks funders to send the message “if you want the money, then build trustworthy AI!”. (shrink)

Motivated by the question of responsible AI and value alignment, I seek to offer a uniquely Foucauldian reconstruction of the problem as the emergence of an ethical subject in a disciplinary setting. This reconstruction contrasts with the strictly human-oriented programme typical to current scholarship that often views technology in instrumental terms. With this in mind, I problematise the concept of a technological subjectivity through an exploration of various aspects of ChatGPT in light of Foucault’s work, arguing that current systems (...) lack the reflexivity and self-formative characteristics inherent in the notion of the subject. By drawing upon a recent dialogue between Foucault and phenomenology, I suggest four techno-philosophical desiderata that would address the gaps in this search for a technological subjectivity: embodied self-care, embodied intentionality, imagination and reflexivity. Thus I propose that advanced AI be reconceptualised as a subject capable of “technical” self-crafting and reflexive self-conduct, opening new pathways to grasp the intertwinement of the human and the artificial. This reconceptualization holds the potential to render future AI technology more transparent and responsible in the circulation of knowledge, care and power. (shrink)

Abstract: One approach to alignment with human values in AI and robotics is to engineer artiTicial systems isomorphic with human beings. The idea is that robots so designed may autonomously align with human values through similar developmental processes, to realize project ideal conditions through iterative interaction with social and object environments just as humans do, such as are expressed in narratives and life stories. One persistent problem with human value orientation is that different human beings champion different values as (...) ideal, meaning that the values to which an AI should be aligned are ambiguous. Prior work considered human development of purpose and source of meaning in life in terms of project ideal conditions, in effect establishing lifelong value orientations according to which intermediate situations are evaluated. The present work compares views on motivating values from St. Augustine and from popular cognitive science. These accounts are described as divergent error theories which present to their proponents as mutually exclusive yet accurate accounts of personal experience due to differential development of variable innate potentials. SpeciTically, the hypothesis proposed is that differential development of spindle neural projections establishes enduring connections between prior established relatively immediate routine processes entrained during childhood and prioritized in popular cognitive science, and later developing higher-level social and self-processes entrained during adolescence and emphasized in Augustine's account, with these projections hard-wiring lifelong motivating value orientations more or less inaccessible to modiTication through material interaction. Robot experiments informed by this study may evaluate variable value orientation by design, with for example autonomous robots developing motivating associations with temporally distal project ideal conditions through love for humanity, as described by Augustine, and others pursuing adaptive Tit to passing norms consistent with popular contemporary accounts. (shrink)