By the early 2020s, emotional artificial intelligence will become increasingly present in everyday objects and practices such as assistants, cars, games, mobile phones, wearables, toys, marketing, insurance, policing, education and border controls. There is also keen interest in using these technologies to regulate and optimize the emotional experiences of spaces, such as workplaces, hospitals, prisons, classrooms, travel infrastructures, restaurants, retail and chain stores. Developers frequently claim that their applications do not identify people. Taking the claim at face value, (...) this paper asks, what are the privacy implications of emotional AI practices that do not identify individuals? To investigate privacy perspectives on soft non-identifying emotional AI, the paper draws upon the following: over 100 interviews with the emotion detection industry, legal community, policy-makers, regulators and NGOs interested in privacy; a workshop with stakeholders to design ethical codes for using data about emotions; a UK survey of 2068 citizens on feelings about emotion capture technologies. It finds a weak consensus among social stakeholders on the need for privacy, this driven by different interests and motivations. Given this weak consensus, it concludes that there exists a limited window of opportunity to societally agree principles of practice regarding privacy and the use of data about emotions. (shrink)

This paper interrogates the growing pervasiveness of affect recognition tools as an emerging layer human-centric automated management in the global workplace. While vendors tout the neoliberal incentives of emotion-recognition technology as a pre-eminent tool of workplace wellness, we argue that emotional AI recalibrates the horizons of capital not by expanding outward into the consumer realm (like surveillance capitalism). Rather, as a new genus of digital Taylorism, it turns inward, passing through the corporeal exterior to extract greater surplus value and (...) managerial control from the affective states of workers. Thus, empathic surveillance signals a profound shift in the ontology of human labor relations. In the emotionally quantified workplace, employees are no longer simply seen as physical capital, but conduits of actuarial and statistical intelligence gleaned from their most intimate subjective states. As a result, affect-driven automated management means that priority is often given to actuarial rather than human-centered managerial decisions. (shrink)

This paper assesses leading Japanese philosophical thought since the onset of Japan’s modernity: namely, from the Meiji Restoration onwards. It argues that there are lessons of global value for AI ethics to be found from examining leading Japanese philosophers of modernity and ethics, each of whom engaged closely with Western philosophical traditions. Turning to these philosophers allows us to advance from what are broadly individualistically and Western-oriented ethical debates regarding emergent technologies that function in relation to AI, by introducing notions (...) of community, wholeness, sincerity, and heart. With reference to AI that pertains to profile, judge, learn, and interact with human emotion, this paper contends that Japan itself may internally make better use of historic indigenous ethical thought, especially as it applies to question of data and relationships with technology; but also that externally Western and global ethical discussion regarding emerging technologies will find valuable insights from Japan. The paper concludes by distilling from Japanese philosophers of modernity four ethical suggestions, or spices, in relation to emerging technological contexts for Japan’s national AI policies and international fora, such as standards development and global AI ethics policymaking. (shrink)

Biometric technologies are becoming more pervasive in the workplace, augmenting managerial processes such as hiring, monitoring and terminating employees. Until recently, these devices consisted mainly of GPS tools that track location, software that scrutinizes browser activity and keyboard strokes, and heat/motion sensors that monitor workstation presence. Today, however, a new generation of biometric devices has emerged that can sense, read, monitor and evaluate the affective state of a worker. More popularly known by its commercial moniker, Emotional AI, the technology (...) stems from advancements in affective computing. But whereas previous generations of biometric monitoring targeted the exterior physical body of the worker, concurrent with the writings of Foucault and Hardt, we argue that emotion-recognition tools signal a far more invasive disciplinary gaze that exposes and makes vulnerable the inner regions of the worker-self. Our paper explores attitudes towards empathic surveillance by analyzing a survey of 1015 responses of future job-seekers from 48 countries with Bayesian statistics. Our findings reveal affect tools, left unregulated in the workplace, may lead to heightened stress and anxiety among disadvantaged ethnicities, gender and income class. We also discuss a stark cross-cultural discrepancy whereby East Asians, compared to Western subjects, are more likely to profess a trusting attitude toward EAI-enabled automated management. While this emerging technology is driven by neoliberal incentives to optimize the worksite and increase productivity, ultimately, empathic surveillance may create more problems in terms of algorithmic bias, opaque decisionism, and the erosion of employment relations. Thus, this paper nuances and extends emerging literature on emotion-sensing technologies in the workplace, particularly through its highly original cross-cultural study. (shrink)

Emotional AI is an emerging technology used to make probabilistic predictions about the emotional states of people using data sources, such as facial (micro)-movements, body language, vocal tone or the choice of words. The performance of such systems is heavily debated and so are the underlying scientific methods that serve as the basis for many such technologies. In this article I will engage with this new technology, and with the debates and literature that surround it. Working at the (...) intersection of criminology, policing, surveillance and the study of emotional AI this paper explores and offers a framework of understanding the various issues that these technologies present particularly to liberal democracies. I argue that these technologies should not be deployed within public spaces because there is only a very weak evidence-base as to their effectiveness in a policing and security context, and even more importantly represent a major intrusion to people’s private lives and also represent a worrying extension of policing power because of the possibility that intentions and attitudes may be inferred. Further to this, the danger in the use of such invasive surveillance for the purpose of policing and crime prevention in urban spaces is that it potentially leads to a highly regulated and control-oriented society. I argue that emotion recognition has severe impacts on the right to the city by not only undertaking surveillance of existing situations but also making inferences and probabilistic predictions about future events as well as emotions and intentions. (shrink)

This paper explores the ways in which Cordwainer Smith’s short story “Under Old Earth” problematizes emotions, who/what has them, and who/what is granted moral status. Most importantly, however, “Under Old Earth” questions the primacy of happiness in human society, especially where happiness is understood as the absence of other (negative) emotions. As such, “Under Old Earth” challenges the notion, widely held in contemporary ethics, that our moral obligation to one another is mediated through the goal of the attainment of happiness. (...) Through this challenge, Smith’s short story speaks directly to the current understanding of emotion in the field of AI, and complicates what the goal of affective computing should be. (shrink)

In recent years, research in robotics and artificial intelligence (AI) has made rapid progress. It is expected that robots and AI will play a part in the field of nursing and their role might broaden in the future. However, there are areas of nursing practice that cannot or should not be entrusted to robots and AI, because nursing is a highly humane practice, and therefore, there would, perhaps, be some practices that should not be replicated by robots or AI. Therefore, (...) this paper focuses on several ethical concepts (advocacy, accountability, cooperation, and caring) that are considered important in nursing practice, and examines whether it is possible to implement these ethical concepts in robots and AI by analyzing the concepts and the current state of robotics and AI technology. Advocacy: Among the components of advocacy, safeguarding and apprising can be more easily implemented, while elements that require emotional communication with patients, such as valuing and mediating, are difficult to implement. Accountability: Robotic nurses with explainable AI have a certain level of accountability. However, the concept of explanation has problems of infinite regression and attribution of responsibility. Cooperation: If robot nurses are recognized as members of a community, they require the same cooperation as human nurses. Caring: More difficulties are expected in care-receiving than in caregiving. However, the concept of caring itself is ambiguous and should be explored further. Accordingly, our analysis suggests that, although some difficulties can be expected in each of these concepts, it cannot be said that it is impossible to implement them in robots and AI. However, even if it were possible to implement these functions in the future, further study is needed to determine whether such robots or AI should be used for nursing care. In such discussions, it will be necessary to involve not only ethicists and nurses but also an array of society members. (shrink)

The prospective study follows college students after the 9/11 attacks. Based on evidence and trauma-related theories, and guided by reports on positive and negative reactions and meaning-related actions among Americans after 9/11, we explored the seemingly contradictory, yet meaning-related pathways to posttraumatic growth and posttraumatic stress disorder symptoms , indicating the sense of deep interconnectedness and deep conflict. The final model showed that 9/11 emotional turmoil triggered processes of assimilation, as indicated in pathways between prayer coping and perceived spiritual (...) and social support, and of accommodation, as indicated in the pathway of spiritual struggle. Both pathways were directly associated with PTG in the follow-up. Perceived spiritual and social support contributed to lower levels of PTSD symptoms, whereas spiritual struggle had an opposite impact on symptoms; both effects were mediated through optimism but in opposite directions. The study suggests the interplay of complex crisis-related phenomena following meaning-laden collective trauma that presented existential challenges, involving coping, optimism, deep conflict, and deep interconnectedness. (shrink)

Personal AI is a new type of AI companion, distinct from the prevailing forms of AI companionship. Instead of playing a narrow and well-defined social role, like friend, lover, caretaker, or colleague, with a set of pre-determined responses and behaviors, Personal AI is engineered to tailor itself to the user, including learning to mirror the user’s unique emotional language and attitudes. This paper identifies two issues with Personal AI. First, like other AI companions, it is deceptive about the presence (...) of their emotions, which undermines the moral value of companionship. Second, Personal AI leads to a distinctly new form of deception concerning the origins of its emotions. Its emotional attitudes appear to belong to it, when in fact they are only reflections of the user. This results in what I dub “emotional bubbles”—the false impression that personal emotions are externally validated—which have at least two troubling implications. First, emotional bubbles prevent us from encountering emotional attitudes that differ from our own, which is likely to cripple emotional growth and the ability to form diverse social and emotional relationships. Second, if we assume, as some philosophers claim, that shared emotions are constitutive of shared values, it follows that Personal AI subverts joint moral deliberation. Users believe their personal values are externally validated, when they are only validated by themselves. Because of the absence of technovirtues able to handle this problem, I suggest that we proceed very cautiously with the development and marketing of Personal AI. (shrink)

Human emotional expressions can communicate the emotional state of the expresser, but they can also communicate appeals to perceivers. For example, sadness expressions such as crying request perceivers to aid and support, and anger expressions such as shouting urge perceivers to back off. Some contemporary artificial intelligence (AI) systems can mimic human emotional expressions in a (more or less) realistic way, and they are progressively being integrated into our daily lives. How should we respond to them? Do (...) we have reasons to reply to the appeals made by AI emotional expressions? In this paper, we examine the conditions under which AI emotional expressions could give us prudential or even moral reasons to change our behavior. We argue that these conditions do not depend on whether the emotional expression is genuine or not, but rather on the presence of features some of which can be implemented in emotive AI given our current level of technological development. We extract recommendations and warnings for the development of emotive AI. (shrink)

The existential risk posed by ubiquitous artificial intelligence (AI) is a subject of frequent discussion with descriptions of the prospect of misuse, the fear of mass destruction, and the singularity. In this paper I address an under-explored category of existential risk posed by AI, namely emotional risk. Values are a main source of emotions. By challenging some of our most essential values, AI systems are therefore likely to expose us to emotional risks such as loss of care and (...) loss of meaning. Part one presents a study of a leading bank in Germany where an AI system was implemented to replace humans in decision-making processes. Part two explains why humans actively make use of values to make decisions. Part three shows the connection between values and emotions. Part four relates parts two and three to the bank study by giving concrete examples of how the employees saw their roles as workers, and how the relationships to their customers changed emotionally after the AI system was implemented to make decisions. (shrink)

This study investigates the relationship between the usage of artificial intelligence (AI), specifically AI-based personalized recommendations (AI-PR), and its impact on consumers’ green product awareness (GPA) during online shopping. Recognizing that AI’s influence alone may not significantly affect green buying behaviours, we introduce emotional intelligence (EI) as a moderator to better understand human impact when using AI. The research framework is built upon the Interactive Recommendation Agents and the SOBC framework. A self-administered survey was conducted with 403 respondents, and (...) the data were analysed using Partial Least Squares Structural Equation Modelling (PLS-SEM). The findings reveal that AI and AI-PR positively impact GPA, with partial mediation observed among the variables. Furthermore, emotional intelligence positively moderates the relationship between perceived AI usage and GPA. Specifically, individuals with higher emotional intelligence exhibit a stronger positive association between AI usage and GPA. Conversely, emotional intelligence negatively moderates the relationship between AI-based personalized recommendations and GPA, indicating that higher EI weakens the relationship between AI-PR and GPA, while lower EI strengthens it. The implications of this study are significant for marketers and retailers aiming to utilize AI-based personalized recommendations to encourage green behaviour and contribute to a sustainable future. These insights can help them understand the complex dynamics between AI, emotional intelligence, green product awareness, and behavioural intentions. (shrink)

Imagine a man chatting with his AI girlfriend app. He looks at his smartphone and says, ‘Finally, I'm being understood’. Is he deceiving himself? Is there anything morally wrong with it? The human tendency to anthropomorphize AI is well established, and the popularity of AI companions is growing. This article answers three questions: (1) How can being charmed by AI's simulated emotions be considered self‐deception? (2) Why might we have an obligation to avoid harmless self‐deception? (3) When is self‐deception in (...) emotional relationships with AI morally questionable, and can it be blameworthy? Regarding question 1, I describe being seduced by AI's simulated emotions as self‐deception, where desires bias beliefs. In response to question 2, I outline two ways to justify a prima facie obligation to avoid harmless self‐deception – instrumental and autotelic. For question 3, I highlight crucial factors to consider in assessing the blameworthiness of harmless self‐deception, such as the emotional and cognitive competences of the self‐deceiver, reasons for self‐deception, and its consequences for one's predispositions, self‐image, and other people. Moreover, I argue that the ethical requirement to avoid self‐deception does not easily translate into attributing blame to others for being self‐deceived. (shrink)

One relatively neglected challenge in ethical artificial intelligence (AI) design is ensuring that AI systems invite a degree of emotional and moral concern appropriate to their moral standing. Although experts generally agree that current AI chatbots are not sentient to any meaningful degree, these systems can already provoke substantial attachment and sometimes intense emotional responses in users. Furthermore, rapid advances in AI technology could soon create AIs of plausibly debatable sentience and moral standing, at least by some relevant (...) definitions. Morally confusing AI systems create unfortunate ethical dilemmas for the owners and users of those systems, since it is unclear how those systems ethically should be treated. I argue here that, to the extent possible, we should avoid creating AI systems whose sentience or moral standing is unclear and that AI systems should be designed so as to invite appropriate emotional responses in ordinary users. (shrink)

One of the main difficulties in assessing artificial intelligence (AI) is the tendency for people to anthropomorphise it. This becomes particularly problematic when we attach human moral activities to AI. For example, the European Commission’s High-level Expert Group on AI (HLEG) have adopted the position that we should establish a relationship of trust with AI and should cultivate trustworthy AI (HLEG AI Ethics guidelines for trustworthy AI, 2019, p. 35). Trust is one of the most important and defining activities in (...) human relationships, so proposing that AI should be trusted, is a very serious claim. This paper will show that AI cannot be something that has the capacity to be trusted according to the most prevalent definitions of trust because it does not possess emotive states or can be held responsible for their actions—requirements of the affective and normative accounts of trust. While AI meets all of the requirements of the rational account of trust, it will be shown that this is not actually a type of trust at all, but is instead, a form of reliance. Ultimately, even complex machines such as AI should not be viewed as trustworthy as this undermines the value of interpersonal trust, anthropomorphises AI, and diverts responsibility from those developing and using them. (shrink)

This article examines the social acceptability and governance of emotional artificial intelligence in children’s toys and other child-oriented devices. To explore this, it conducts interviews with stakeholders with a professional interest in emotional AI, toys, children and policy to consider implications of the usage of emotional AI in children’s toys and services. It also conducts a demographically representative UK national survey to ascertain parental perspectives on networked toys that utilise data about emotions. The article highlights disquiet about (...) the evolution of generational unfairness, that encompasses injustices regarding the datafication of childhood, manipulation, parental vulnerability, synthetic personalities, child and parental media literacy, and need for improved governance. It concludes with practical recommendations for regulators and the toy industry. (shrink)

The increasing integration of artificial intelligence and virtual reality in online learning platforms has raised profound philosophical and theological questions regarding human cognition, emotional engagement, and spiritual formation. However, current virtual learning environments often overlook the significant role of emotional and affective factors in shaping user experience, particularly in the context of religious education and moral development. This study explores an emotionally designed virtual interactive learning platform, incorporating a philosophical inquiry into how technology-mediated learning affects human emotions, ethical (...) reasoning, and spiritual growth. Utilizing a MacBERT-BiSRU-AT emotion classification model, this research examines user emotional responses to virtual learning interactions, achieving an F1 score of 91.34% with efficient processing time. Findings reveal that emotionally responsive design enhances learners’ engagement, mitigates negative emotional experiences, and improves satisfaction, with over 85% of users expressing positive responses. The implications of this study extend beyond technological advancements, addressing deeper philosophical concerns about the nature of learning, digital embodiment, and the intersection of artificial intelligence with religious pedagogy. By considering users’ emotional and spiritual needs, the study suggests that emotionally intelligent design can enrich online religious education, fostering a more profound engagement with theological discourse, moral reflection, and community-based learning. Future research should further explore how AI-driven emotional design influences religious cognition, ethical development, and the metaphysical dimensions of virtual learning environments. (shrink)

Some have claimed that since machines lack emotional “qualia”, or conscious experiences of emotion, machine intelligence will fall short of human intelligence. I examine this objection, ultimately finding it unpersuasive. I first discuss recent work on emotion that suggests that emotion plays various roles in cognition. I then raise the following question: are phenomenal experiences of emotion an essential or necessary component of the performance of these cognitive abilities? I then sharpen the question by distinguishing between four possible positions (...) one might take. I reject one of these four positions largely on empirical grounds. But the remaining three positions all suggest that even if emotional qualia play an important role in human cognition, emotional qualia are not essential to the performance of these cognitive abilities in principle, so, e.g., a machine that lacks emotional qualia might still be able to perform them. (shrink)

Emotions are a central aspect of communication. Consequently, emotion analysis (EA) is a rapidly growing field in natural language processing (NLP). However, there is no consensus on scope, direction, or methods. In this paper, we conduct a thorough review of 154 relevant NLP publications from the last decade. Based on this review, we address four different questions: (1) How are EA tasks defined in NLP? (2) What are the most prominent emotion frameworks and which emotions are modeled? (3) Is the (...) subjectivity of emotions considered in terms of demographics and cultural factors? and (4) What are the primary NLP applications for EA? We take stock of trends in EA and tasks, emotion frameworks used, existing datasets, methods, and applications. We then discuss four lacunae: (1) the absence of demographic and cultural aspects does not account for the variation in how emotions are perceived, but instead assumes they are universally experienced in the same manner; (2) the poor fit of emotion categories from the two main emotion theories to the task; (3) the lack of standardized EA terminology hinders gap identification, comparison, and future goals; and (4) the absence of interdisciplinary research isolates EA from insights in other fields. Our work will enable more focused research into EA and a more holistic approach to modeling emotions in NLP. (shrink)

By looking at the politics of classification within machine learning systems, this article demonstrates why the automated interpretation of images is an inherently social and political project. We begin by asking what work images do in computer vision systems, and what is meant by the claim that computers can “recognize” an image? Next, we look at the method for introducing images into computer systems and look at how taxonomies order the foundational concepts that will determine how a system interprets the (...) world. Then we turn to the question of labeling: how humans tell computers which words will relate to a given image. What is at stake in the way AI systems use these labels to classify humans, including by race, gender, emotions, ability, sexuality, and personality? Finally, we turn to the purposes that computer vision is meant to serve in our society—the judgments, choices, and consequences of providing computers with these capacities. Methodologically, we call this an archeology of datasets: studying the material layers of training images and labels, cataloguing the principles and values by which taxonomies are constructed, and analyzing how these taxonomies create the parameters of intelligibility for an AI system. By doing this, we can critically engage with the underlying politics and values of a system, and analyze which normative patterns of life are assumed, supported, and reproduced. (shrink)

The proliferation of Generative Artificial Intelligence (Generative AI) has led to an increased reliance on AI‐generated content for designing and deploying digital health interventions. While generative AI has the potential to facilitate and automate healthcare, there are concerns that AI‐generated content and AI‐generated health advice could trigger, perpetuate, or exacerbate prior traumatic experiences among vulnerable populations. In this discussion article, I examined how generative‐AI‐powered digital health interventions could trigger, perpetuate, or exacerbate emotional trauma among vulnerable populations who rely on (...) digital health interventions as complementary or alternative sources of seeking health services or information. I then proposed actionable strategies for mitigating AI‐generated trauma in the context of digital health interventions. The arguments raised in this article are expected to shift the focus of AI practitioners against prioritizing dominant narratives in AI algorithms into seriously considering the needs of vulnerable minority groups who are at the greatest risk for trauma but are often invisible in AI data sets, AI algorithms, and their resultant technologies. (shrink)

The psychological contract refers to the implicit and subjective beliefs regarding a reciprocal exchange agreement, predominantly examined between employees and employers. While contemporary contract research is investigating a wider range of exchanges employees may hold, such as with team members and clients, it remains silent on a rapidly emerging form of workplace relationship: employees’ increasing engagement with technically, socially, and emotionally sophisticated forms of artificially intelligent (AI) technologies. In this paper we examine social robots (also termed humanoid robots) as likely (...) future psychological contract partners for human employees, given these entities transform notions of workplace technology from being a tool to being an active partner. We first overview the increasing role of robots in the workplace, particularly through the advent of sociable AI, and synthesize the literature on human–robot interaction. We then develop an account of a human-social robot psychological contract and zoom in on the implications of this exchange for the enactment of reciprocity. Given the future focused nature of our work we utilize a thought experiment, a commonly used form of conceptual and mental model reasoning, to expand on our theorizing. We then outline potential implications of human-social robot psychological contracts and offer a range of pathways for future research. (shrink)

This paper demonstrates that with increased philosophical scrutiny regarding the absurdity of life and meaningful living, the development of “Emotionalized AI” falls under ethical considerations. In this context, contemporary philosophical discussions on the meaning of life—as articulated by thinkers like Thomas Nagel, Joshua Seachris, Thaddeus Metz, and Susan Wolf—intersect with recent years’ reviews of AI ethics, particularly those related to meaningful relationships. The main analysis is conducted by explaining a philosophical perspective on meaningful life and revisiting AI ethics reviews based (...) on it while accepting a superficial understanding of “Emotionalized AI” to avoid metaphysical disputes. In this regard, the questions raised about AI ethics, considering the specialized literature in recent years, are first addressed. Similarly, the discussion focuses on a specific type of ethical concern regarding AI development: human relationships. It is then explained how “love” and, more generally, emotional relationships can be an important resource of life’s meaningfulness. Subsequently, an argument is presented showing that the development of emotionalized AI, by limiting our access to “emotional relationships,” poses a threat to life’s meaningfulness and is therefore ethically unjustifiable. To reinforce this view, I also use some near-reality dystopian scenarios for the AI era. Finally, I re-evaluate the conclusion based on a four-step process for assessing the meaningfulness gap created by emotionalized AI. Applying this process also confirms the result. (shrink)

Emotions play a significant role in human relations, decision-making, and the motivation to act on those decisions. There are ongoing attempts to use artificial intelligence (AI) to read human emotions, and to predict human behavior or actions that may follow those emotions. However, a person’s emotions cannot be easily identified, measured, and evaluated by others, including automated machines and algorithms run by AI. The ethics of emotional AI is under research and this study has examined the emotional variables (...) as well as the perception of emotional AI in two large random groups of college students in an international university in Japan, with a heavy representation of Japanese, Indonesian, Korean, Chinese, Thai, Vietnamese, and other Asian nationalities. Surveys with multiple close-ended questions and an open-ended essay question regarding emotional AI were administered for quantitative and qualitative analysis, respectively. The results demonstrate how ethically questionable results may be obtained through affective computing and by searching for correlations in a variety of factors in collected data to classify individuals into certain categories and thus aggravate bias and discrimination. Nevertheless, the qualitative study of students’ essays shows a rather optimistic view over the use of emotional AI, which helps underscore the need to increase awareness about the ethical pitfalls of AI technologies in the complex field of human emotions. (shrink)

Our rapidly increasing reliance on frictionless AI interactions may increase cognitive and emotional distance, thereby letting our adaptive resilience slacken and our ethical virtues atrophy from disuse. Many trends already well underway involve the offloading of cognitive, emotional, and ethical labor to AI software in myriad social, civil, personal, and professional contexts. Gradually, we may lose the inclination and capacity to engage in critically reflective thought, making us more cognitively and emotionally vulnerable and thus more anxious and prone (...) to manipulation from false news, deceptive advertising, and political rhetoric. In this article, I consider the overarching features of this problem and provide a framework to help AI designers tackle it through system enhancements in smartphones and other products and services in the burgeoning internet of things (IoT) marketplace. The framework is informed by two ideas: psychologist Daniel Kahneman’s cognitive dual process theory and moral self-awareness theory, a four-level model of moral identity that I developed with Benjamin M. Cole. (shrink)

This article critically examines the paradigm shift from the attention economy to the intimacy economy—a market system where personal and emotional data are exchanged for customized experiences that cater to individual emotional and psychological needs. It explores how AI transforms these personal and emotional inputs into services, thereby raising essential questions about the authenticity of digital interactions and the potential commodification of intimate experiences. The study delineates the roles of human–computer interaction and AI in deepening personal connections, (...) significantly impacting emotional dynamics, and underscores AI’s role in various applications, from healthcare to grief tech, highlighting both enhancements in emotional connections and potential disruptions to genuine human interactions. An AI-mediated framework (AMIE) is introduced to assess how AI reshapes these connections and the overall digital society through personalized interactions. This framework explores the interplay between human emotions and AI-generated responses within the new Avatar Sphere, emphasizing the necessity for regulatory measures to safeguard digital identities, recognize emotional data as intellectual property, and maintain system transparency. It highlights the critical need for maintaining genuine human interactions and advocates for context-aware consent, continuous monitoring, and cross-cultural considerations to foster ethical AI practices. Leveraging blockchain technology and decentralized autonomous organizations, the framework proposes methods to enhance individual control over emotional data, mitigating the commodification risks. The findings contribute to ongoing discussions on AI ethics, digital privacy, and the future of human-AI interactions, providing valuable insights for cultivating a responsible intimacy economy. (shrink)

Artificial Intelligence (AI) Agents are often compared to psychopaths in popular news articles. The headlines are ‘eye-catching’, but the questions of what this analogy means or why it matters are hardly answered. The aim of this paper is to take this popular analogy ‘seriously’. By that, I mean two things. First, I aim to explore the scope of this analogy, i.e. to identify and analyse the shared properties of AI agents and psychopaths, namely, their lack of moral emotions and their (...) capacity for instrumental rationality. Second, I aim to examine the impact of the analogy. I argue that both agents, as ‘amoral calculators’, present the perfect candidates to revisit two long-standing debates on moral and criminal responsibility, regarding the necessity of moral emotions for ‘moral-agent-capacity responsibility’ and the necessity of ‘moral-agent-capacity responsibility’ for criminal responsibility. Finally, cross-examining the debates on the moral and criminal responsibility of psychopaths and AI agents is instructive and revealing. Instructive since the moral and legal treatment of psychopaths can be telling about the future treatment of AI agents (and vice versa) and revealing since it makes explicit our often-implicit philosophical commitments on the criteria of moral agency and the overarching purpose of criminal law. (shrink)

Emotions are an integral part of human cognition and they guide not only our understanding of the world but also our actions within it. As such, whether we soothe or flame an emotion is not inconsequential. Recent work in conversational AI has focused on responding empathetically to users, validating and soothing their emotions without a real basis. This AI-aided emotional regulation can have negative consequences for users and society, tending towards a one-noted happiness defined as only the absence of (...) "negative" emotions. We argue that we must carefully consider whether and how to respond to users' emotions. (shrink)

Attempts to engineer a generally intelligent artificial agent have yet to meet with success, largely due to the (intercontext) frame problem. Given that humans are able to solve this problem on a daily basis, one strategy for making progress in AI is to look for disanalogies between humans and computers that might account for the difference. It has become popular to appeal to the emotions as the means by which the frame problem is solved in human agents. The purpose of (...) this paper is to evaluate the tenability of this proposal, with a primary focus on Dylan Evans’ search hypothesis and Antonio Damasio’s somatic marker hypothesis. I will argue that while the emotions plausibly help solve the intracontext frame problem, they do not function to solve or help solve the intercontext frame problem, as they are themselves subject to contextual variability. (shrink)

The development of AI tools, such as large language models and speech emotion and facial expression recognition systems, has raised new ethical concerns about AI’s impact on human relationships. While much of the debate has focused on human-AI relationships, less attention has been devoted to another class of ethical issues, which arise when AI mediates human-to-human relationships. This paper opens the debate on these issues by analyzing the case of romantic relationships, particularly those in which one partner uses AI tools, (...) such as ChatGPT, to resolve a conflict and apologize. After reviewing some possible, non-exhaustive, explanations for the moral wrongness of using AI tools in such cases, I introduce the notion of second-person authenticity: a form of authenticity that is assessed by the other person in the relationship (e.g., a partner). I then argue that at least some actions within romantic relationships should respect a standard of authentic conduct since the value of such actions depends on who actually performs them and not only on the quality of the outcome produced. Therefore, using AI tools in such circumstances may prevent agents from meeting this standard. I conclude by suggesting that the proposed theoretical framework could also apply to other human-to-human relationships, such as the doctor-patient relationship, when these are mediated by AI; I offer some preliminary reflections on such applications. (shrink)

This paper investigates whether AI can possess wisdom, a complex and deeply human capacity. We adopt an interdisciplinary approach, drawing on philosophy, 4E cognition, Material Engagement Theory, and engaged epistemology, to argue that wisdom is a dynamic force unfolding through meaningful life experiences and resonant interactions with the world. Central to our discussion is Rosa's concept of resonance, essential for fostering personal growth, emotional empathy, as well as existential and bodily connectedness to the world and its unfolding into an (...) uncertain future. Despite the advanced capabilities of AI/Large Language Models (LLMs), we demonstrate their inability to embody wisdom due to their lack of deep sense-making, embodied interaction, and meaningful engagement with the world and future potentials. We emphasize the need for cultivating human wisdom to ensure technological advancements enrich rather than diminish the human experience, advocating for a balanced and reflective human-technology relationship. (shrink)

Recent developments in AI, especially the spectacular success of Large Language models, have instigated renewed questioning of what remains distinctively human. As AI stands poised to take over more and more human tasks, what is left that distinguishes humans? One way we might identify a humanlike intelligence would be when we detect it telling lies. Yet AIs lack both the intention and the motivation to truly tell lies, instead producing merely bullshit. With neither emotions, embodiment, nor the social awareness that (...) leads to a theory of mind, AIs lack the internal referents on which to judge truth or falsity. When we are deceived by our computers, we need to look for the hidden agent who benefits from the deception. (shrink)

Critics currently argue that applied ethics approaches to artificial intelligence (AI) are too principles-oriented and entail a theory–practice gap. Several applied ethical approaches try to prevent such a gap by conceptually translating ethical theory into practice. In this article, we explore how the currently most prominent approaches of AI ethics translate ethics into practice. Therefore, we examine three approaches to applied AI ethics: the embedded ethics approach, the ethically aligned approach, and the Value Sensitive Design (VSD) approach. We analyze each (...) of these three approaches by asking how they understand and conceptualize theory and practice. We outline the conceptual strengths as well as their shortcomings: an embedded ethics approach is context-oriented but risks being biased by it; ethically aligned approaches are principles-oriented but lack justification theories to deal with trade-offs between competing principles; and the interdisciplinary Value Sensitive Design approach is based on stakeholder values but needs linkage to political, legal, or social governance aspects. Against this background, we develop a meta-framework for applied AI ethics conceptions with three dimensions. Based on critical theory, we suggest these dimensions as starting points to critically reflect on the conceptualization of theory and practice. We claim, first, that the inclusion of the dimension of affects and emotions in the ethical decision-making process stimulates reflections on vulnerabilities, experiences of disregard, and marginalization already within the AI development process. Second, we derive from our analysis that considering the dimension of justifying normative background theories provides both standards and criteria as well as guidance for prioritizing or evaluating competing principles in cases of conflict. Third, we argue that reflecting the governance dimension in ethical decision-making is an important factor to reveal power structures as well as to realize ethical AI and its application because this dimension seeks to combine social, legal, technical, and political concerns. This meta-framework can thus serve as a reflective tool for understanding, mapping, and assessing the theory–practice conceptualizations within AI ethics approaches to address and overcome their blind spots. (shrink)

This is a reply to comments on my paper Cog, a Humanoid Robot, and the Questions of the Image of God; one was written by Mary Gerhart and Allan Melvin Russell, and another one by Helmut Reich. I will start with the suggested analogy of the relationship between God and us and the one between us and the humanoid robot Cog and will show why this analogy is not helpful for the dialogue between theology and artificial intelligence (AI). Such a (...) dialogue can succeed only if both our fascination for humanoids and our fear of them are equally accepted. Any avoidance of these emotions, as well as any rejection of the possibility that Cog might one day be humanlike, destroy the dialogue. The interpretation of both scientific theories and religious metaphors as stories replaces seemingly “rational” arguments with the confession of the respective commitments to a body of stories and opens up a space for exchange and friendship between AI‐researchers and theologians—an option that usually remains closed. (shrink)

: In the analytic tradition of the philosophy of emotions the folk notion of adequacy is understood with regard to – at least four – different questions, viz. a moral question, a prudential question, an epistemic question, and a fittingness question. Usually, the fittingness question is treated as being the central one. I have some doubts concerning this assessment, particularly when it comes to substantial – interpersonal or cultural – controversies about whether a specific emotional response is adequate or (...) whether a specific event deserves the emotional responses it triggers. To approach these matters, I recommend first doing without the established distinctions, for they may prematurely tempt us into assessing the adequacy of emotional responses in terms of one of these categories thereby overlooking other features that deserve attention. Instead, I will start with the folk notion of adequacy and then refine stepwise the conceptual landscape to get closer to what the crucial issues of adequacy are. Keywords: Philosophy of Emotions; Emotional Atmospheres; Existential Feelings; Adequacy of Emotional Responses L’adeguatezza delle emozioni e dei sentimenti esistenziali Riassunto: All’interno della tradizione analitica della filosofia delle emozioni la nozione comune di adeguatezza è presa in considerazione in relazione ad almeno quattro diversi problemi: un primo è di carattere morale ; un secondo riguarda la discrezionalità ; un terzo è di tipo epistemico e un quarto concerne l’ appropriatezza. Di solito, all’appropriatezza viene riservata maggiore importanza. Personalmente nutro diversi dubbi su questa valutazione, in particolare quando si tratta di dirimere questioni sostanziali – interpersonali o culturali –, se una data risposta emotiva è adeguata o se un dato evento merita la risposta emotiva che ha innescato. Per affrontare questi problemi, mi sento in primo luogo di suggerire di fare a meno delle distinzioni maggiormente in uso, poiché possono spiprematuramente a valutare l’adeguatezza delle risposte emotive nei termini una delle categorie già menzionate, trascurando quindi altre proprietà degne di attenzione. Intendo invece partire dalla nozione di adeguatezza propria del senso comune e poi gradualmente raffinare il panorama concettuale, per approssimarmi ai problemi fondamentali dell’adeguatezza. Parole chiave: Filosofia delle emozioni; Atmosfere emotive; Sentimenti esistenziali; Adeguatezza delle risposte emotive. (shrink)

In this article we tackle the core question of machine emotion research – “Can machines have emotions?” – in the context of “social robots”, a new class of machines designed to function as “social partners” for humans. Our aim, however, is not to provide an answer to the question “Can robots have emotions?” Rather we argue that the “robotics of emotion” moves us to reformulate it into a different one – “Can robots affectively coordinate with humans?” Developing a series of (...) arguments relevant to theory of emotion, philosophy of AI, and the epistemology of synthetic models, we argue that the answer to this different question is positive, and that it lays grounds for an innovative ethical approach to emotional robots. This ethical project, which we introduced elsewhere as “synthetic ethics”, rejects the diffused ethical condemnation of emotional robots as “cheating” technology. Synthetic ethics focuses not on an ideological refusal, but on the concrete sustainability of the emerging mixed human-robot social ecologies. On this basis, in contrast to a purely negative ethical approach to social robotics it promotes an analytical case by case ethical inquiry into the type of human flourishing that can result from human-robot affective coordination. (shrink)

Linguistic competence is, among other things, the cognitive ability of using words appropriately. Following this notion, being a competent language user cannot be merely simulated because pretending to use words appropriately already is using words appropriately. Since current AI already can use words appropriately it may be that AI already has linguistic competence instead of merely simulating it. However, in this article I argue that there is at least one domain of linguistic competence that can be simulated: grasping lexical effects. (...) Since grasping lexical effects presupposes an ability which AI lacks, namely feeling emotions, I argue that AI can only grasp the cognitive part of words but not their lexical effects. AI can only simulate to grasp lexical effects. Hence, lexical effects are a domain of linguistic competence AI cannot grasp. (shrink)

Artificial intelligence (AI) is becoming part of the everyday. During this transition, people’s intention to use AI technologies is still unclear and emotions such as fear are influencing it. In this paper, we focus on autonomous cars to first verify empirically the extent to which people fear AI and then examine the impact that fear has on their intention to use AI-driven vehicles. Our research is based on a systematic survey and it reveals that while individuals are largely afraid of (...) cars that are driven by AI, they are nonetheless willing to adopt this technology as soon as possible. To explain this tension, we extend our analysis beyond just fear and show that people also believe that AI-driven cars will generate many individual, urban and global benefits. Subsequently, we employ our empirical findings as the foundations of a theoretical framework meant to illustrate the main factors that people ponder when they consider the use of AI tech. In addition to offering a comprehensive theoretical framework for the study of AI technology acceptance, this paper provides a nuanced understanding of the tension that exists between the fear and adoption of AI, capturing what exactly people fear and intend to do. (shrink)

This article discusses the application of artificially intelligent robots within eldercare and explores a series of ethical considerations, including the challenges that AI (Artificial Intelligence) technology poses to traditional Chinese Confucian filial piety. From the perspective of Confucian ethics, the paper argues that robots cannot adequately fulfill duties of care. Due to their detachment from personal relationships and interactions, the “emotions” of AI robots are merely performative reactions in different situations, rather than actual emotional abilities. No matter how “humanized” (...) robots become, it is difficult to establish genuine empathy and a meaningful relationship with them for this reason. Even so, we acknowledge that AI robots are a significant tool in managing the demands of elder care and the growth of care poverty, and as such, we attempt to outline some parameters within which care robotics could be acceptable within a Confucian ethical system. Finally, the paper discusses the social impact and ethical considerations brought on by the interaction between humans and machines. It is observed that the relationship between humans and technology has always had both utopian and dystopian aspects, and robotic elder care is no exception. AI caregiver robots will likely become a part of elder care, and the transformation of these robots from “service providers” to “companions” seems inevitable. In light of this, the application of AI-augmented robotic elder care will also eventually change our understanding of interpersonal relationships and traditional requirements of filial piety. (shrink)

Organizations are increasingly adopting “Emotional AI” to monitor and influence employee emotions, aiming to create more empathetic workplaces. However, we argue that automating empathy risks fostering empathy skepticism, alienating employees, exacerbating mental health issues, and eroding trust. We call on organizations to address the root causes of negative workplace emotions and leverage AI as a tool to complement—rather than replace—empathy, fostering workplaces that genuinely prioritize care and trust.