According to an intriguing though somewhat enigmatic line of thought first proposed by Jonathan Bennett, if humanity went extinct any time soon this would be unfortunate because important business would be left unfinished. This line of thought remains largely unexplored. I offer an interpretation of the idea that captures its intuitive appeal, is consistent with plausible constraints, and makes it non-redundant to other views in the literature. The resulting view contrasts with a welfare-promotion perspective, according to which extinction would be (...) bad because it would come at a massive opportunity cost in terms of feasibly achievable welfare over the lifetime of our species. (shrink)

When our choice affects some other person and the outcome is unknown, it has been argued that we should defer to their risk attitude, if known, or else default to use of a risk avoidant risk function. This, in turn, has been claimed to require the use of a risk avoidant risk function when making decisions that primarily affect future people, and to decrease the desirability of efforts to prevent human extinction, owing to the significant risks associated with (...) continued human survival. I raise objections to the claim that respect for others’ risk attitudes requires risk avoidance when choosing for future generations. In particular, I argue that there is no known principle of interpersonal aggregation that yields acceptable results in variable population contexts and is consistent with a plausible ideal of respect for others’ risk attitudes in fixed population cases. GPI Working Paper No . 20-2022. (shrink)

Humanity stands at a precipice. -/- Our species could survive for millions of generations — enough time to end disease, poverty, and injustice; to reach new heights of flourishing. But this vast future is at risk. With the advent of nuclear weapons, humanity entered a new age, gaining the power to destroy ourselves, without the wisdom to ensure we won’t. Since then, these dangers have only multiplied, from climate change to engineered pandemics and unaligned artificial intelligence. If we do (...) not act fast to reach a place of safety, it may soon be too late. -/- The Precipice explores the science behind the risks we face. It puts them in the context of the greater story of humanity: showing how ending these risks is among the most pressing moral issues of our time. And it points the way forward, to the actions and strategies we can take today to safeguard humanity’s future. (shrink)

> Future people count. There could be a lot of them. We can make their lives better. > > -- William MacAskill, What We Owe The Future > > [Longtermism is] quite possibly the most dangerous secular belief system in the world today. > > -- Émile P. Torres, Against Longtermism Philosophers,1 2 psychologists,3 4 politicians5 and even some tech billionaires6 have sounded the alarm about artificial intelligence (AI) and the dangers it may pose to the (...) long-term future of humanity. Some believe it poses an existential risk (X-Risk)7 to our species, potentially causing our extinction or bringing about the collapse of human civilisation as we know it. The above quote from philosopher Will MacAskill captures the key tenets of ‘longtermism’, an ethical standpoint that places the onus on current generations to prevent AI-related—and other—X-Risks for the sake of people living in the future.1 2 Developing from an adjacent social movement commonly associated with utilitarian philosophy, ‘effective altruism’, longtermism has amassed following of its own. Its supporters argue that preventing X-Risks is at least as morally significant as addressing current challenges like global poverty. However, critics are concerned that such a distant-future focus will sideline efforts to tackle the many pressing moral issues facing humanity now.8 9 Indeed, according to ‘strong’ longtermism,10 future needs arguably should take precedence over present ones. In essence, the claim is that there is greater expected utility to allocating available resources to prevent human extinction in the future than there is to focusing on present lives, since doing so stands to benefit the incalculably large number of people in later generations who will far outweigh existing populations.1 Taken to the extreme, this view suggests it would be morally permissible, or even required, to actively neglect, harm or destroy large swathes of humanity as …. (shrink)

Since the release of ChatGPT, there has been a lot of debate about whether AI systems pose an existential risk to humanity. This paper develops a general framework for thinking about the existential risk of AI systems. We analyze a two-premise argument that AI systems pose a threat to humanity. Premise one: AI systems will become extremely powerful. Premise two: if AI systems become extremely powerful, they will destroy humanity. We use these two premises to construct a taxonomy (...) of ‘survival stories’, in which humanity survives into the far future. In each survival story, one of the two premises fails. Either scientific barriers prevent AI systems from becoming extremely powerful; or humanity bans research into AI systems, thereby preventing them from becoming extremely powerful; or extremely powerful AI systems do not destroy humanity, because their goals prevent them from doing so; or extremely powerful AI systems do not destroy humanity, because we can reliably detect and disable systems that have the goal of doing so. We argue that different survival stories face different challenges. We also argue that different survival stories motivate different responses to the threats from AI. Finally, we use our taxonomy to produce rough estimates of ‘P(doom)’, the probability that humanity will be destroyed by AI. (shrink)

Because of accelerating technological progress, humankind may be rapidly approaching a critical phase in its career. In addition to well-known threats such as nuclear holocaust, the propects of radically transforming technologies like nanotech systems and machine intelligence present us with unprecedented opportunities and risks. Our future, and whether we will have a future at all, may well be determined by how we deal with these challenges. In the case of radically transforming technologies, a better understanding of the transition (...) dynamics from a human to a "posthuman" society is needed. Of particular importance is to know where the pitfalls are: the ways in which things could go terminally wrong. While we have had long exposure to various personal, local, and endurable global hazards, this paper analyzes a recently emerging category: that of existential risks. These are threats that could case our extinction or destroy the potential of Earth - originating intelligent life. Some of these threats are relatively well known while others, including some of the gravest, have gone almost unrecognized. Existential risks have a cluster of features that make ordinary risk management ineffective. A final section of this paper discusses several ethical and policy implications. A clearer understanding of the threat picture will enable us to formulate better strategies. (shrink)

This paper provides a critique of Bostrom’s concern with existential risks, a critique which relies on Adorno and Horkheimer’s interpretation of the Enlightenment. Their interpretation is used to elicit the inner contradictions of transhumanist thought and to show the invalid premises on which it is based. By first outlining Bostrom’s position this paper argues that transhumanism reverts to myth in its attempt to surpass the human condition. Bostrom’s argument is based on three pillars, Maxipok, Parfitian population ethics and (...) a universal notion of general human values. By attempting to transcend the human condition, to achieve post-humanity, transhumanism reverts to myth. Thus, the aim of this paper is to provide a critical examination of transhumanism which elicits its tacit contradictions. It will also be argued that transhumanism’s focus on a universal, all-encompassing, notion of humanity neglects any concern with actual lived lives. This absence is problematic because it clearly shows that there is a discrepancy, between transhumanism’s claimed concern for all of humanity and the practical implications of proposing a universal notion of humanity. This paper will conclude, that transhumanism’s lack of concern with actual lives is due to its universal and totalising gestures. Gestures which allow for universal claims such as general values or Earth-originating intelligent life. (shrink)

What moral reasons, if any, do we have to prevent the extinction of humanity? In “Unfinished Business,” Jonathan Knutzen argues that certain further developments in culture would make our history more ‘collectively meaningful,’ and that premature extinction would be bad because it would close off that possibility. Here, I critically examine this proposal. I argue that if collective meaningfulness is analogous to individual meaningfulness, then our meaning-based reasons to prevent the extinction of humanity are substantially different from the reasons discussed (...) by Knutzen. (shrink)

We outline an argument favoring voluntary moral bioenhancement as a response to existential risks humanity exposes itself to. We consider this type of enhancement a solution to the antithesis between the extinction of humanity and the imperative of humanity to survive at any cost. By opting for voluntary moral bioenhancement; we refrain from advocating illiberal or even totalitarian strategies that would allegedly help humanity preserve itself. We argue that such strategies; by encroaching upon the freedom of individuals; already inflict (...) a degree of existential harm on human beings. We also give some pointers as to the desirable direction for morally enhanced post-personhood. (shrink)

Large-scale, self-sufficient space colonization is a plausible means of efficiently reducing existential risks and ensuring our long-term survival. But humanity is by and large myopic, and as an intergenerational global public good, existential risk reduction is systematically undervalued, hampered by intergenerational discounting. This paper explores how these issues apply to space colonization, arguing that the motivational and psychological barriers to space colonization are a special—and especially strong—case of a more general problem. The upshot is not that large-scale, self-sufficient (...) space colonization will never occur, but that, absent institutional change, the conditions under which it is most likely to occur are precisely those conditions where the threat of suffering risks might be most high. (shrink)

Brad Hooker argues that the cost of inculcating in everyone the prevent disaster rule places a limit on its demandingness. My aim in this article is show that this is not true of existential risk reduction. However, this does not spell trouble for the reason that removing persistent global harms significantly improves our long-run chances of survival. We can expect things to get better, not worse, for our population.

Recently many methods for reducing the risk of human extinction have been suggested, including building refuges underground and in space. Here we will discuss the perspective of using military nuclear submarines or their derivatives to ensure the survival of a small portion of humanity who will be able to rebuild human civilization after a large catastrophe. We will show that it is a very cost-effective way to build refuges, and viable solutions exist for various budgets and timeframes. Nuclear (...) submarines are surface independent, and could provide energy, oxygen, fresh water and perhaps even food for their inhabitants for years. They are able to withstand close nuclear explosions and radiation. They are able to maintain isolation from biological attacks and most known weapons. They already exist and need only small adaptation to be used as refuges. But building refuges is only “Plan B” of existential risk preparation; it is better to eliminate such risks than try to survive them. (shrink)

This chapter makes the case for strong longtermism: the claim that, in many situations, impact on the long-run future is the most important feature of our actions. Our case begins with the observation that an astronomical number of people could exist in the aeons to come. Even on conservative estimates, the expected future population is enormous. We then add a moral claim: all the consequences of our actions matter. In particular, the moral importance of what happens does (...) not depend on when it happens. That pushes us toward strong longtermism. We then address a few potential concerns, the first of which is that it is impossible to have any sufficiently predictable influence on the course of the long-run future. We argue that this is not true. Some actions can reasonably be expected to improve humanity’s long-term prospects. These include reducing the risk of human extinction, preventing climate change, guiding the development of artificial intelligence, and investing funds for later use. We end by arguing that these actions are more than just extremely effective ways to do good. Since the benefits of longtermist efforts are large and the personal costs are comparatively small, we are morally required to take up these efforts. (shrink)

The study of social, ethical, and political issues from a naturalistic perspective has been pervasive in social sciences and the humanities in the last decades. This articulation of empirical research with philosophical and normative reflection is increasingly getting attention in academic circles and the public spheres, given the prevalence of urgent needs and challenges that society is facing on a global scale. The contemporary world is full of challenges or what some philosophers have called ‘existential risks’ to humanity. Nuclear (...) wars, natural and/or engineered pandemics, climate change, global totalitarianism, and emergent technologies, are some of the challenges mentioned by many authors devoted to the study of long-term potential risks for the survival of the human species. However, while there are many reasons to be a pessimist about the future of humanity, there is also a lot of ink wasted in showing that the world today is far better than we had in the past and also that there have been many instances of social and moral progress in the last centuries. Some authors have even claimed that “we have made more progress over the last 100 years than in the first 100,000”. According to ‘our world in data’, some of the most relevant advances of humanity in the last centuries have to do with the reduction of poverty and/or inequality, better health and security, the decline of violence and war, and the development of democracy and institutional mechanisms to guarantee individual freedom and rights. Yet, there is a more concrete concern that our world faces nowadays, and requires a particularly detailed analysis. In a few words, the advance in information and communication technologies, the huge volume of people migration because of prevalent instances of global inequality, and the reality of an interconnected world that just mirrors an extensive history of massive human intergroup relations open the question about the feasibility of social and moral progress in the contexts of diverse, multicultural, and unequal societies. This makes it crucial to reflect not only on whether human societies may keep advancing on a path of social progress, but how this can be achieved by addressing the challenges of inequality and potential intergroup conflict. This concern closely relates to the further consolidation of democratic values and the achievement of a more generalized consideration of the equal moral value of human individuals far from group, ethnic, or collective distinctions. This vision of progress follows the moral convictions that have been considered right and worthy to be pursued by most ethical theories explored by philosophers and social scientists so far. This work aims to contribute to the study of the mechanisms and feasibility of social and moral progress grounded on the precept of human equality and is guided by the idea that social and moral progress in a plural world requires the accomplishment of a more democratic and tolerant society. These forms of progress have already been defined by philosophers Allen Buchanan and Russel Powell as inclusivist forms of moral progress. According to these authors, inclusivist moral progress is featured by the transformation of moral judgments, motivations, and norms guided by the rejection of “group-based restrictions on moral standing and moral status” (2018). The work of Buchanan and Powell is fundamental for the argumentation developed in this work since they have pioneered a more complex debate in the field of evolutionary social sciences and philosophical anthropology. In brief, these authors have inquired how human nature, conceptualized by the advances in evolutionary thinking and the empirical sciences, is related to our moral convictions and the possibilities that we have to establish certain forms of social and moral progress. The work of Buchanan and Powell contributes to the debate on moral progress by incorporating empirically-informed concepts and arguments. With this objective in mind, they develop their discussion around a generalized tension between exclusivist and inclusivist forms of moral progress. This tension revolves around the question of whether and to what extent human moral concerns, motivations, and norms, are limited or bounded by social identities, group thinking, or collective values. This question is at the center of this work. In other words, I aim to answer whether and to what extent the nature of our social and moral cognition hinders or enables the development of inclusivist forms of morality and moral progress. As I mentioned so far, the distinction between exclusivist and inclusivist forms of morality has a practical implication and facilitates the reflection on the mechanisms and feasibility of concrete forms of progress based on the precept of human equality, which is the road to progress that we should follow according to most ethical theories. The distinction between exclusivist and inclusivist morality is not used here as a form of summarizing highly divergent descriptive approaches to morality, but as a way to address the practical implications of morality research, particularly around the question of the feasibility of social forms of progress based on the precept of equality and tolerance. My objective then has less to do with offering a conceptualization of human morality or with the proposition of a new ethical perspective. Instead, I aim to identify, from an empirically informed perspective, what enables the achievement of egalitarian forms of social progress, to subsequently offer a normative reflection about what we should do if we want to accomplish such inclusivist forms of progress. This task is of great relevance, having in mind that some theoretical proposals on the matter have claimed that human prosocial behavior and morality are constrained by exclusivist fixed tendencies as a product of our evolutionary history. According to these views, human morality is a group-centered cognitive and behavioral expression that has severe limits in its scope and extent, which hinders in the end the emergence of inclusivist forms of moral concerns and behaviors. These claims align with the philosophical intuition about the ‘dissociation of empathy’, or the fact that human individuals tend to express empathy and sympathy preferably for members of their group whereas their empathic and moral concern and recognition for outgroup individuals tend to be reduced or eliminated in real-life social interactions. Multiple theorists have expressed, however, that these assumptions are wrong for empirical and philosophical reasons, which results in a serious challenge to the conception of human morality as exclusivist by nature. Furthermore, simple intuition and practical experience constantly show us that human individuals can develop and endorse a broad spectrum of moral considerations and adopt an arguably inclusivist stance concerning the kind of persons or agents that deserve moral concern. Moreover, and adopting a historical perspective, the expansion of cultural and commercial exchanges, the consolidation of democracy, the development of welfare states, or the emergence of institutions based on the concept of universal human rights, have partially driven human progress based on the impartial concern for the well-being of persons. These historical instances of cultural evolution and institutional design have shown that it is feasible to promote behaviors and attitudes grounded in impersonal prosocial and inclusivist principles. This optimistic perspective, however, still begs the question of the exact relationship between social, cultural, and historical contexts, and the emergence of certain behavioral expressions of inclusivist morality. This is crucial since a great number of human individuals around the world still do not make part of these instances of social progress, and some authors have suggested that such instances of social and moral progress are particularly favored by the contexts of the WEIRD world, that is the world of Western, Educated, Industrialized, Rich, and Democratic societies. Some of these approaches consider moral inclusivity a ‘luxury good’ and follow the ‘dependency thesis’ according to which the expression of moral inclusivity depends on the presence of very concrete material conditions that trigger inclusivist and impersonal prosocial tendencies that human beings are not able to express in threatening contexts without physical security. This thesis is problematic for various reasons and has led to incomplete explanations about the enabling and causal factors of inclusivist moral progress, as well as unsatisfactory suggestions about what we should do to foster the accomplishment of more generalized instances of progress based on inclusivist concerns. In an attempt to contribute to this debate, the main objective of this thesis is to offer an alternative proposal about the nature of human morality and a more detailed reflection on what we should do if we want to further accomplish such instances of progress. More precisely, this work adopts a particular developmental and constructivist vision of human morality to answer the question of how feasible it is to reach generalized forms of social progress based on the precepts of moral inclusivity. Furthermore, it offers some ideas about what are the most adequate and favorable conditions to foster such progress both at the individual and the collective level. At this point, a preliminary clarification of some starting points is necessary. In the first place, and contrary to what other authors have proposed when debating moral progress from an empirically-based approach, my perspective adopts a radical developmental and constructivist perspective, following particularly the conceptual and empirical ideas of the social-domain theory in moral psychology. From this perspective, morality is a domain of normative and social knowledge that develops and emerges in human interactions. Moral concerns, judgments, and norms are not only the result of the expression of innate adaptive intuitions, nor the product of social conformity and a simple process of mirroring the normativity that communities have established to organize their social life. Human individuals, instead, construct moral concerns and judgments during their situated interactions. More specifically, the constructivist tradition claims that we need to research how moral concerns, judgments, and reasons emerge in social interactions and how human beings construct the content of their moral knowledge in the first place. Based on these ideas, morality is defined here as the result of a constructive process through which human individuals, by the means of constant interactions on a background of mutual respect and recognition, develop evaluative concerns about how we ought to treat others, and how to ‘care’ and ‘respect’ others’ well-being, needs, and vulnerabilities. This developmental-constructivist theory of human psychology contributes to an alternative understanding of the origins of moral variability and informs a very particular normative stance about the most desirable contexts needed for the urgent accomplishment of inclusivist forms of morality in contemporary societies. This is so given that the very same definition of morality adopted presupposes the feature of inclusivity. The main objective of this work is then to argue, based on a developmental perspective, what are the most favorable conditions to foster the development of moral inclusivity and what we should do to accomplish concrete instances of moral progress that truly help us to overcome our most urgent moral challenges. The whole thesis is organized as follows. In part I, I show how received visions about human social evolution have informed conceptions of morality as an adaptive function evolved to solve the challenges of cooperative life. Based on these assumptions, some theories argue that human morality is limited by the boundaries of group thinking, and always obeys the mandates of collective interests and the will of the majority. However, this conception has problematic implications from a normative and ethical perspective. The aim of Chapter 1 is to present this introductory debate in detail. In chapter 2, I consider a relevant amount of scientific evidence that helps to consolidate a more optimistic and inclusivist perspective about the starting conditions of human prosocial behavior. Later, I will summarize some ideas offered to explain the possible evolutionary mechanisms that led to the consolidation of these inclusivist capacities. However, I will also show how evolutionary theories that share the same adaptationist foundations but initially seem relevant to explain the feasibility of moral inclusivity, result incomplete when considering the complexity of moral behavior and its ethical implications. Part II is devoted to presenting a developmental alternative to the adaptationist versions around the starting conditions of morality. In this section, I first summarize the epistemological, conceptual, and empirical basis of a constructivist perspective already proposed by several authors in the last decades. This is the goal of chapter 3. More precisely, I adopt the conceptualization of morality established by the social-domain theory school and clarify its conception of the situated development of moral concerns, motivations, and norms. All these ideas are essential for my argumentation on the feasibility of inclusivist moral progress offered later. Chapter 4 provides a more detailed idea of constructivist thinking about moral development, by analyzing one particular dimension: the development of moral concerns for fairness and the sense of justice in distributive dilemmas. In Part III, I assume a constructivist perspective on morality to contribute to the philosophy of moral progress. In particular, I discuss the implications of such a perspective for the conceptualization of human moral nature and discuss what are the most favorable conditions for moral inclusivity to emerge. Answering that question is of enormous relevance since favoring the transition of human societies towards more inclusivist forms of moral concern and behavior supposes a more generalized accomplishment of our most urgent moral convictions. Previous developmental or culturally-sensible attempts to explore this issue are already on the market. As mentioned before, most authors have approached the topic by situating the cultural evolution and development of (inclusivist) moral progress in the WEIRD world. For instance, some of these authors claim that very concrete material conditions (mostly present in the Western World) have been necessary for triggering inclusivist and impersonal prosocial tendencies that human beings are not able to express in threatening contexts without physical security. Despite their interest in explaining inclusivist morality and the feasibility of inclusivist forms of progress following a naturalistic and culturally situated approach, these theories are problematic for two main reasons. In the first place, most of the authors that explain moral progress tend to focus exclusively on the consolidation of liberal values in the Western world and the underlying adoption of laws, social-epistemic practices, and public narratives about tolerance and inclusivity, as unequivocal signs of inclusivist morality. Nevertheless, there is an enormous distance between the institutional and public support of inclusivist moral values or the construction of public images of tolerance and inclusivity, and the real materialization of impersonal moralities in situated social interactions. Secondly, the variability of human moral behavior is far bigger than most of the theories in moral psychology and moral progress tend to assume. Accordingly, inclusivist and exclusivist forms of morality may develop simultaneously in similar or divergent geographical and socio-ecological environments, including those with harsh conditions such as competition for scarce resources, potential intergroup conflict, and risks of disease transmission. In addition, harsh environmental conditions are usually present in almost all societies around the globe, and the presence of socio-ecological conditions such as physical security and prosperity, predominant in highly resourced populations, do not prevent the existence of exclusivist moral tendencies. My model explores a different perspective on the relationships between cultural, institutional, and historical factors, and the situated development of human morality. This supposes a clear distinction from previous approaches to the matter since it offers a different view on what we should do to pursue further instances of inclusivist social progress. In the fifth chapter, I first analyze the proposals of some defenders of the ‘dependency’ theory. The attention is focused on the ‘Evolutionary Developmental Model’ proposed by Buchanan and Powell (2018). Based on a critique of this approach, I will argue that to explain the evolution and development of human morality in its multiple manifestations, it is essential to put focus on social interactions. This also involves a different perspective on human-environment relations that in line with a constructivist worldview makes an innovative contribution to the philosophy of moral behavior and moral progress. Later, in the sixth chapter, I will define my alternative relational view of inclusivist morality and I will offer my view on how to establish favorable conditions that make inclusivist morality possible. Complementing the constructivist approach, my conception is based on the essential role of equality and individuality and on how moral inclusivity is favored if moral concerns do not develop driven by the accomplishment of group-centered, binding, and communitarian concerns and values. In brief, my relational model suggests that the accomplishment of moral inclusivity not only depends on the overall achievement of certain levels of physical security or economic prosperity. Instead, its development is favored by transforming the interactional contexts in which moral concerns towards autonomy and individuality emerge. In arguing so, I will need to start with a critique of contemporary situated and embodied approaches in ethics. Then, I will show how societies guided by the accomplishment of group-centered values of power, dominance, and relational inequality tend to express exclusivist moral behaviors, while individualistic societies tend to be more inclusivist. Starting from this diagnosis, my final attempt is then to reflect on which are the best conditions for the moral valuation of individuality to emerge and what we should do to promote the accomplishment of moral inclusivity at the societal level. (shrink)

This article proposes a metaethical route from neo-Aristotelian naturalism, as developed in particular by Philippa Foot, to virtue-ethical longtermism. It argues that the metaethical assumptions of neo-Aristotelian naturalism inherently imply that a valid description of the life-form of a species must satisfy a formal requirement of internal sustainability. The elements of a valid life-form description then serve as a normative standard. Given that humans have the ability to influence the fate of future generations and know about their influence, (...) this article posits that our future as a species becomes a part of the description of the human life-form. Consequently, in our case, the sustainability requirement is extended into the future, leading to a minimal and humanist version of longtermism. As in Rosalind Hursthouse’s naturalistic virtue ethics, the focus of the proposed virtue-ethical longtermism remains on developing virtues as the only reliable way for the agent to achieve the good life. However, since what can count as virtue must be compatible with a life-form description that, taken as a whole, facilitates the long-term survival of our species, this position still implies a genuine longtermism, although it contrasts with the quantifying and maximizing logic of consequentialism. (shrink)

Is the overall value of a world just the sum of values contributed by each value-bearing entity in that world? Additively separable axiologies (like total utilitarianism, prioritarianism, and critical level views) say 'yes', but non-additive axiologies (like average utilitarianism, rank-discounted utilitarianism, and variable value views) say 'no'. This distinction appears to be practically important: among other things, additive axiologies generally assign great importance to large changes in population size, and therefore tend to strongly prioritize the long-term survival of humanity over (...) the interests of the present generation. Non-additive axiologies, on the other hand, need not assign great importance to large changes in population size. We show, however, that when there is a large enough `background population' unaffected by our choices, a wide range of non-additive axiologies converge in their implications with additive axiologies—for instance, average utilitarianism converges with critical-level utilitarianism and various egalitarian theories converge with prioritarianism. We further argue that real-world background populations may be large enough to make these limit results practically significant. This means that arguments from the scale of potential future populations for the astronomical importance of avoiding existential catastrophe, and other arguments in practical ethics that seem to presuppose additive separability, may succeed in practice whether or not we accept additive separability as a basic axiological principle. (shrink)

Endocrine-disrupting chemicals have been identified as posing risks to reproductive health and may have intergenerational effects. However, responses to the potential harms they pose frequently rely on medicalised understandings of the body and normative gender identities. This article develops an intersectional feminist framework of intergenerational justice in response to the potential risks posed by endocrine-disrupting chemicals. We examine critiques of endocrine disruptors from feminist, critical disability and queer standpoints, and explore issues of race and class in exposures. We argue that (...) responding to the risks posed by endocrine disruptors such as brominated flame retardants (BFRs) and phthalates requires developing a theory of intergenerational justice that recognises relationality and transcorporeality, and that also recognises harm in terms of suffering, not in terms of difference. (shrink)

This paper examines a recent argument in favour of strong precautionary action—possibly including working to hasten human extinction—on the basis of a decision-theoretic view that accommodates the risk-attitudes of all affected while giving more weight to the more risk-averse attitudes. First, we dispute the need to take into account other people’s attitudes towards risk at all. Second we argue that a version of the non-identity problem undermines the case for doing so in the context of future people. Lastly, (...) we suggest that we should not work to hasten human extinction, even if significant risk aversion is warranted. (shrink)

Longtermism is the view that the most urgent global priorities, and those to which we should devote the largest portion of our resources, are those that focus on (i) ensuring a long future for humanity, and perhaps sentient or intelligent life more generally, and (ii) improving the quality of the lives that inhabit that long future. While it is by no means the only one, the argument most commonly given for this conclusion is that these interventions have (...) greater expected goodness per unit of resource devoted to them than each of the other available interventions, including those that focus on the health and well-being of the current population. In this paper, I argue that, even if we grant the consequentialist ethics upon which this argument depends, and even if we grant one of the axiologies that are typically paired with that ethics to give the argument, we are not morally required to choose an option that maximises expected utility; indeed, we might not even be permitted to do so. Instead, I will argue, if the argument's consequentialism is correct, we should choose using a decision theory that is sensitive to risk, and allows us to give greater weight to worse-case outcomes than expected utility theory does. And, I will show, such decision theories do not always recommend longtermist interventions. Indeed, sometimes, they recommend exactly the opposite: sometimes, they recommend hastening human extinction. Many, though not all, will take this as a reductio of the consequentialism or the axiology of the argument. I remain agnostic on the conclusion we should draw. (shrink)

How can one build AI systems such that they pursue the goals their designers want them to pursue? This is the alignment problem. Numerous authors have raised concerns that, as research advances and systems become more powerful over time, misalignment might lead to catastrophic outcomes, perhaps even to the extinction or permanent disempowerment of humanity. In this paper, I analyze the severity of this risk based on current instances of misalignment. More specifically, I argue that contemporary large language models and (...) game-playing agents are sometimes misaligned. These cases suggest that misalignment tends to have a variety of features: misalignment can be hard to detect, predict and remedy, it does not depend on a specific architecture or training paradigm, it tends to diminish a system’s usefulness and it is the default outcome of creating AI via machine learning. Subsequently, based on these features, I show that the risk of AI alignment magnifies with respect to more capable systems. Not only might more capable systems cause more harm when misaligned, aligning them should be expected to be more difficult than aligning current AI. (shrink)

Recenze knihy:Toby Ord, The Precipice. Existential Risk and the Future of Humanity. Bloomsbury Publishing, 2020, 480 s.

Longtermists have argued that humanity should significantly increase its efforts to prevent catastrophes like nuclear wars, pandemics, and AI disasters. But one prominent longtermist argument overshoots this conclusion: the argument also implies that humanity should reduce the risk of existential catastrophe even at extreme cost to the present generation. This overshoot means that democratic governments cannot use the longtermist argument to guide their catastrophe policy. In this paper, we show that the case for preventing catastrophe does not depend on (...) longtermism. Standard cost-benefit analysis implies that governments should spend much more on reducing catastrophic risk. We argue that a government catastrophe policy guided by cost-benefit analysis should be the goal of longtermists in the political sphere. This policy would be democratically acceptable, and it would reduce existential risk by almost as much as a strong longtermist policy. (shrink)

In this paper, we argue that our moral concern for future well-being should reduce over time due to important practical considerations about how humans interact with spacetime. After surveying several of these considerations (around equality, special duties, existential contingency, and overlapping moral concern) we develop a set of core principles that can both explain their moral significance and highlight why this is inherently bound up with our relationship with spacetime. These relate to the equitable distribution of (1) moral (...) concern in a universe that we can only causally affect in one temporal direction; (2) access to the benefits from using spacetime as a resource; and (3) the burdens of care given (1) and (2). We conclude by considering the practical implications of our argument and find that, while it is often assumed that a preference for present over future well-being weakens the case for existential risk mitigation, this likely is not the case. (shrink)

Nuclear weapons are different from every other type of weapons technology. Their awesome destructive potential and the unparalleled consequences of their use oblige us to think critically about the ethics of nuclear possession, planning, and use. Joe Nye has given the ethics of nuclear weapons deep consideration. He posits that we have a basic moral obligation to future generations to preserve roughly equal access to important values, including equal chances of survival, and proposes criteria for achieving conditional or “just (...) deterrence” to minimize the risk of nuclear use and help preserve these values. While Nye's conditions are laudable, they are not achievable. They rely on flawed assumptions about the nature of nuclear weapons and the inherent risks of the nuclear deterrence system. Since the Cold War ended, the strategy and practice of nuclear deterrence has grown riskier, more urgent, more dangerous, and less stable. It is time to rethink how we manage nuclear risks. A new nuclear security system must be built on the design principle that the consequences of system failure cannot threaten to end or fundamentally disrupt civilization. We owe the future a new nuclear security strategy that can prevent an existential global nuclear event. (shrink)

When our choice affects some other person and the outcome is unknown, it has been argued that we should defer to their risk attitude, if known, or else default to use of a risk‐avoidant risk function. This, in turn, has been claimed to require the use of a risk‐avoidant risk function when making decisions that primarily affect future people, and to decrease the desirability of efforts to prevent human extinction, owing to the significant risks associated with continued (...) class='Hi'>human survival. I raise objections to the claim that respect for others' risk attitudes requires risk‐avoidance when choosing for future generations. In particular, I argue that there is no known principle of interpersonal aggregation that yields acceptable results in variable population contexts and is consistent with a plausible ideal of respect for others' risk attitudes in fixed population cases. (shrink)

Philosophy confronts two existential crises: the threats to its existence from scientists like Stephen Hawking who claim that philosophy is dead; and the threat to life itself from catastrophic climate change. The essay’s first theoretical part critiques Nietzsche’s claim that philosophy’s primary function is to guarantee the future of life. The essay’s second practical part claims that philosophy must meet the challenge of life’s extinction through a revised model for ethics in education. Taking its start from recent conceptualizations (...) of philosophy as a ‘field discipline,’ this part considers African American models for delivering ethics in education to under-resourced communities. The third part shows how these theoretical models may be actualized in the practice of an African field philosopher. (shrink)

In The Expanse, the future of humanity is constantly at stake. In The Expanse vestiges of an ancient alien civilization with incredibly advanced technology have been found—which eventually permits human interstellar expansion through the gates. James Lenman argues that, even if we agree that biodiversity is a good thing, it only means that it's good that there should be natural diversity while life exists on Earth. While we might not be facing interplanetary war or the unpredictable consequences of (...) ancient alien technology, many serious dangers face the human species today. We are quite resilient and have survived numerous disasters in the past, including floods, earthquakes, famine, pandemics, and wars. Small population size and restricted distribution range are factors that increase a species’ extinction risk. Interplanetary expansion could potentially bring about speciation, which means the separation of Homo sapiens into different species. (shrink)

Are we in imminent danger of extinction? Yes, we probably are, argues John Leslie in his chilling account of the dangers facing the human race as we approach the second millenium. The End of the World is a sobering assessment of the many disasters that scientists have predicted and speculated on as leading to apocalypse. In the first comprehensive survey, potential catastrophes - ranging from deadly diseases to high-energy physics experiments - are explored to help us understand the risks. (...) One of the greatest threats facing humankind, however, is the insurmountable fact that we are a relatively young species, a risk which is at the heart of the 'Doomsday Argument'. This argument, if correct, makes the dangers we face more serious than we could have ever imagined. This more than anything makes the arrogance and ignorance of politicians, and indeed philosophers, so disturbing as they continue to ignore the manifest dangers facing future generations. (shrink)

The conventional discourse on existential risks (x-risks) from AI typically focuses on abrupt, dire events caused by advanced AI systems, particularly those that might achieve or surpass human-level intelligence. These events have severe consequences that either lead to human extinction or irreversibly cripple human civilization to a point beyond recovery. This discourse, however, often neglects the serious possibility of AI x-risks manifesting incrementally through a series of smaller yet interconnected disruptions, gradually crossing critical thresholds over time. (...) This paper contrasts the conventional "decisive AI x-risk hypothesis" with an "accumulative AI x-risk hypothesis". While the former envisions an overt AI takeover pathway, characterized by scenarios like uncontrollable superintelligence, the latter suggests a different causal pathway to existential catastrophes. This involves a gradual accumulation of critical AI-induced threats such as severe vulnerabilities and systemic erosion of economic and political structures. The accumulative hypothesis suggests a boiling frog scenario where incremental AI risks slowly converge, undermining societal resilience until a triggering event results in irreversible collapse. Through systems analysis, this paper examines the distinct assumptions differentiating these two hypotheses. It is then argued that the accumulative view can reconcile seemingly incompatible perspectives on AI risks. The implications of differentiating between these causal pathways --- the decisive and the accumulative --- for the governance of AI as well as long-term AI safety are discussed. (shrink)

In a well-known paper, Nick Bostrom presents a confrontation between a fictionalised Blaise Pascal and a mysterious mugger. The mugger persuades Pascal to hand over his wallet by exploiting Pascal's commitment to expected utility maximisation. He does so by offering Pascal an astronomically high reward such that, despite Pascal's low credence in the mugger's truthfulness, the expected utility of accepting the mugging is higher than rejecting it. In this article, I present another sort of high value, low credence mugging. This (...) time, the mugger utilises research on existential risk and the long-term potential of humanity to exploit Pascal's expected-utility-maximising descendant. This mugging is more insidious than Bostrom's original as it relies on plausible facts about the long-term future, as well as realistic credences about how our everyday actions could, albeit with infinitesimally low likelihood, affect the future of humanity. (shrink)

Why, if at all, should we object to economic inequality? Some central arguments – the argument from decreasing marginal utility for example – invoke instrumental reasons and object to inequality because of its effects. Such instrumental arguments, however, often concern only the static effects of inequality and neglect its intertemporal conse- quences. In this article, we address this striking gap and investigate income inequality’s intertemporal consequences, including its potential effects on humanity’s (very) long-term future. Following recent arguments around (...) class='Hi'>future generations and so-called longtermism, those effects might arguably matter more than inequality’s short-term con- sequences. We assess whether we have instrumental reason to reduce economic inequality based on its intertemporal effects in the short, medium, and the very long term. We find a good short and medium-term instrumental case for lower economic inequality. We then argue, somewhat speculatively, that we have instrumental reasons for inequality reduction from a longtermist perspective too, primarily because greater inequality could increase existential risk. We thus have instrumental reasons to reduce inequality, regardless of which time-horizon we take. We then argue that from most consequentialist perspectives, this pro tanto reason also gives us all-things-considered reason. And even across most non-consequentialist views in philosophy, this argument gives us either an all-things-considered or at least weighty pro tanto reason against inequality. (shrink)

Longtermists claim that what we ought to do is mainly determined by how our actions might affect the very long-run future. A natural objection to longtermism is that these effects may be nearly impossible to predict — perhaps so close to impossible that, despite the astronomical importance of the far future, the expected value of our present actions is mainly determined by near-term considerations. This paper aims to precisify and evaluate one version of this epistemic objection to (...) longtermism. To that end, I develop two simple models for comparing ‘longtermist’ and ‘neartermist’ interventions, incorporating the idea that it is harder to make a predictable difference to the further future. These models yield mixed conclusions: if we simply aim to maximize expected value, and don’t mind premising our choices on minuscule probabilities of astronomical payoffs, the case for longtermism looks robust. But on some prima facie plausible empirical worldviews, the expectational superiority of longtermist interventions depends heavily on these ‘Pascalian’ probabilities. So the case for longtermism may depend either on plausible but non-obvious empirical claims or on a tolerance for Pascalian fanaticism. (shrink)

Human civilisation faces a range of existential risks, including nuclear war, runaway climate change and superintelligent artificial intelligence run amok. As we show here with calculations for the New Zealand setting, large numbers of currently living and, especially, future people are potentially threatened by existential risks. A just process for resource allocation demands that we consider future generations but also account for solidarity with the present. Here we consider the various ethical and policy issues involved (...) and make a case for further engagement with the New Zealand public to determine societal values towards future lives and their protection. (shrink)

In this century a number of events could extinguish humanity. The probability of these events may be very low, but the expected value of preventing them could be high, as it represents the value of all future human lives. We review the challenges to studying human extinction risks and, by way of example, estimate the cost effectiveness of preventing extinction-level asteroid impacts.

Jecker et al 1 offer a valuable analysis of risk discussion in relation to Artifical Intelligence (AI) and in the context of longtermism generally, a philosophy prevalent among technocrats and tech billionaires who significantly shape the direction of technological progress in our world. Longtermists accomplish a significant justificatory win, when they use a utilitarian calculation that pits all future humanity against concerns about current humans and societies. By making this argument, they are able to have abstract (and uncertain) (...) benefits for an infinite group of people, outweigh concrete harms for current people—a nifty trick given it also often seems to align with their own personal benefits related to status, power and profit-making. In the AI space, specifically, this has worked well. As Jecker et al 1point out, while AI leaders have spoken openly and loudly about existential risk (X-risk) that conjures thoughts of sci-fi fantasies rather than realities (a strategy that also serves to hype up their work well beyond its current capacity), it fails to engage with concrete and less existential risks that have significant impacts. These risks are inconvenient to development in the Silicon Valley model that requires people to ‘move fast and break things’ rather than proceed carefully and thoughtfully. Not only that but …. (shrink)

“Existential risks” are risks that threaten the destruction of humanity’s long-term potential: risks of nuclear wars, pandemics, supervolcano eruptions, and so on. On standard utilitarianism, it seems, the reduction of such risks should be a key global priority today. Many effective altruists agree with this verdict. But how should the importance of these risks be assessed on a Christian moral theory? In this paper, I begin to answer this question – taking Thomas Aquinas as a reference, and the risks (...) of anthropogenic extinction as an example. I’ll suggest that on a Thomist framework, driving ourselves extinct would constitute a complete failure to fulfil our God-given nature, a radical form of hubris, and a loss of cosmological significance. So the reduction of such risks seems vital for Thomists, or Christians more broadly. Indeed, I end with the thought that these considerations are not parochially Christian, or indeed specifically religious: everyone with a similar view of mankind – as immensely valuable yet fallen and fragile – should agree that existential risks are a pressing moral concern. (shrink)

In recent papers, Elizabeth Finneron-Burns and Johann Frick have both argued that it is not a wrong-making feature of human extinction that it would cause many potential people with lives worth living never to be born, and hence that causing human extinction would be, in at least one way, less wrong than many have thought. In making these arguments, both assume that merely possible future people cannot be harmed by their nonexistence, and thus do not have any (...) claim to be brought into existence. In this paper, we raise objections to their arguments and suggest that there is nothing inherent in the moral theories they put forward that implies future people cannot have this sort of ‘existential’ claim. In doing so, we draw on the work of Derek Parfit, who argued, in a recent paper, that coming into existence benefits a person in that it is ‘good for’ them, even if it is not ‘better for’ them than non-existence. We also find that many of their objections to the view that it is wrong not to bring future people into existence rest on the assumption that, were these people to have claims on us, these must be equivalent to the claims that existing people have to be benefitted. However, we show that Parfit’s work demonstrates how this is not the case. (shrink)

The cumulative impact of enhancement technologies may alter the human species in the very long-term future. In this article, I will start showing how radical genetic enhancements may accelerate the conversion into a novel species. I will also clarify the concepts of ‘biological species’, ‘transhuman’ and ‘posthuman’. Then, I will summarize some ethical arguments for creating a transhuman or posthuman species with a substantially higher level of well-being than the human one. In particular, I will present what (...) I shall call the Principle of the Best Interests of Posthumanity, which states that the enhancement of the human and transhuman species must be directed towards the creation of a posthuman existence that is substantially more valuable than its predecessors. I suggest that human extinction may be considered, within that principle, as one of the best interests of posthumanity. Finally, I will develop three objections that make that principle unattractive and that show that pursuing a full-blown programme of posthuman evolution is ethically flawed. (shrink)

Climate change is widely recognized as a major risk to societies and natural ecosystems but the high end of the risk, i.e., where risks become existential, is poorly framed, defined, and analyzed in the scientific literature. This gap is at odds with the fundamental relevance of existential risks for humanity, and it also limits the ability of scientific communities to engage with emerging debates and narratives about the existential dimension of climate change that have recently gained considerable (...) traction. This paper intends to address this gap by scoping and defining existential risks related to climate change. We first review the context of existential risks and climate change, drawing on research in fields on global catastrophic risks, and on key risks and the so-called Reasons for Concern in the reports of the Intergovernmental Panel on Climate Change. We also consider how existential risks are framed in the civil society climate movement as well as what can be learned in this respect from the COVID-19 crisis. To better frame existential risks in the context of climate change, we propose to define them as those risks that threaten the existence of a subject, where this subject can be an individual person, a community, or nation state or humanity. The threat to their existence is defined by two levels of severity: conditions that threaten (1) survival and (2) basic human needs. A third level, well-being, is commonly not part of the space of existential risks. Our definition covers a range of different scales, which leads us into further defining six analytical dimensions: physical and social processes involved, systems affected, magnitude, spatial scale, timing, and probability of occurrence. In conclusion, we suggest that a clearer and more precise definition and framing of existential risks of climate change such as we offer here facilitates scientific analysis as well societal and political discourse and action. (shrink)

Bioethics, Volume 36, Issue 9, Page 1001-1002, November 2022.

How can the one influence the many? From posing seminal questions about what comprises a human individual, to asking whether human evolution is alive and well, favoring individuals or the species, this work is a daring, up-to-the-minute overview of an urgent, multidisciplinary premise. It explores the extent to which human history provides empirical evidence for the capacity of an individual to exert meaningful suasion over their species, and asks: Can an individual influence the survival of the (...) class='Hi'>human species and the planet? If there are to be cultures of transformation dedicated to seeing us all through the Sixth Extinction Spasm, the Anthropocene, inflicting as little biological havoc as possible, what might such orientations—a collective, widespread biophilia, or reverence for nature—look like? In this powerful work, with a combination of data and direct observation, the authors invite readers to explore how such transformations might resonate throughout the human community; in what ways a person might overcome the seemingly insurmountable environmental tumult our species has unleashed; the clear and salient motives, ethics, aspirations and pragmatic idealism he/she might mirror and embrace in order to effect a profound difference—at the individual level—for all of life and life’s myriad habitats. Chapters illuminate an ambitiously broad digest of research from two-dozen disciplines. Those include ecodynamics, biosemiotics, neural plasticity, anthropology, paleontology and the history of science, among others. All converge upon a set of ethics-based scenarios for mitigating ecological damage to ourselves and other life forms. This highly readable and tightly woven treatise speaks to scientists, students and all those who are concerned about ethical activism and the future of the biosphere. Michael Charles Tobias and Jane Gray Morrison are ecological philosophers and animal liberation activists who have worked for decades to help enrich our understanding of ecosystem dynamics and humanity’s ambiguous presence amid that great orchestra that is nature. (shrink)

This paper examines and analyzes five definitions of ‘existential risk.’ It tentatively adopts a pluralistic approach according to which the definition that scholars employ should depend upon the particular context of use. More specifically, the notion that existential risks are ‘risks of human extinction or civilizational collapse’ is best when communicating with the public, whereas equating existential risks with a ‘significant loss of expected value’ may be the most effective definition for establishing existential risk studies (...) as a legitimate field of scientific and philosophical inquiry. In making these arguments, the present paper hopes to provide a modicum of clarity to foundational issues relating to the central concept of arguably the most important discussion of our times. (shrink)

Journal of Applied Philosophy, EarlyView.

Climate change is often described as an existential risk to the human species, but this terminology has generally been avoided in the climate-justice literature in analytic philosophy. I investigate the source of this disconnect and explore the prospects for incorporating the idea of climate change as an existential risk into debates about climate justice. The concept of existential risk does not feature prominently in these discussions, I suggest, because assumptions that structure ‘ideal’ accounts of climate justice (...) ensure that the prospect of climate change as an extinction-level threat does not arise. Given persistent noncompliance with mitigation duties, however, we have reason to revisit these assumptions. I argue that the most promising way for theories of climate justice to account for the significance of existential risk is to look to the practices of protest and resistance in which a concern about extinction or global catastrophe is frequently given expression. (shrink)

The singularity hypothesis is a radical hypothesis about the future of artificial intelligence on which self-improving artificial agents will quickly become orders of magnitude more intelligent than the average human. Despite the ambitiousness of its claims, the singularity hypothesis has been defended at length by leading philosophers and artificial intelligence researchers. In this paper, I argue that the singularity hypothesis rests on scientifically implausible growth assumptions. I show how leading philosophical defenses of the singularity hypothesis (Chalmers 2010, Bostrom (...) 2014) fail to overcome the case for skepticism. I conclude by drawing out philosophical implications of this discussion for our understanding of consciousness, personal identity, digital minds, existential risk, and ethical longtermism. GPI Working Paper No . 19-2022. (shrink)