Since the release of ChatGPT, there has been a lot of debate about whether AI systems pose an existential risk to humanity. This paper develops a general framework for thinking about the existential risk of AI systems. We analyze a two-premise argument that AI systems pose a threat to humanity. Premise one: AI systems will become extremely powerful. Premise two: if AI systems become extremely powerful, they will destroy humanity. We use these two premises to construct a taxonomy of ‘survival (...) stories’, in which humanity survives into the far future. In each survival story, one of the two premises fails. Either scientific barriers prevent AI systems from becoming extremely powerful; or humanity bans research into AI systems, thereby preventing them from becoming extremely powerful; or extremely powerful AI systems do not destroy humanity, because their goals prevent them from doing so; or extremely powerful AI systems do not destroy humanity, because we can reliably detect and disable systems that have the goal of doing so. We argue that different survival stories face different challenges. We also argue that different survival stories motivate different responses to the threats from AI. Finally, we use our taxonomy to produce rough estimates of ‘P(doom)’, the probability that humanity will be destroyed by AI. (shrink)

Under what conditions would an artificially intelligent system have wellbeing? Despite its clear bearing on the ethics of human interactions with artificial systems, this question has received little direct attention. Because all major theories of wellbeing hold that an individual’s welfare level is partially determined by their mental life, we begin by considering whether artificial systems have mental states. We show that a wide range of theories of mental states, when combined with leading theories of wellbeing, predict that certain existing (...) artificial systems have wellbeing. Along the way, we argue that there are good reasons to believe that artificial systems can have wellbeing even if they are not phenomenally conscious. While we do not claim to demonstrate conclusively that AI systems have wellbeing, we argue that there is a significant probability that some AI systems have or will soon have wellbeing, and that this should lead us to reassess our relationship with the intelligent systems we create. (shrink)

This paper examines the question of whether Large Language Models (LLMs) like ChatGPT possess minds, focusing specifically on whether they have a genuine folk psychology encompassing beliefs, desires, and intentions. We approach this question by investigating two key aspects: internal representations and dispositions to act. First, we survey various philosophical theories of representation, including informational, causal, structural, and teleosemantic accounts, arguing that LLMs satisfy key conditions proposed by each. We draw on recent interpretability research in machine learning to support these (...) claims. Second, we explore whether LLMs exhibit robust dispositions to perform actions, a necessary component of folk psychology. We consider two prominent philosophical traditions, interpretationism and representationalism, to assess LLM action dispositions. While we find evidence suggesting LLMs may satisfy some criteria for having a mind, particularly in game-theoretic environments, we conclude that the data remains inconclusive. Additionally, we reply to several skeptical challenges to LLM folk psychology, including issues of sensory grounding, the "stochastic parrots" argument, and concerns about memorization. Our paper has three main upshots. First, LLMs do have robust internal representations. Second, there is an open question to answer about whether LLMs have robust action dispositions. Third, existing skeptical challenges to LLM representation do not survive philosophical scrutiny. (shrink)

It is generally assumed that existing artificial systems are not phenomenally conscious, and that the construction of phenomenally conscious artificial systems would require significant technological progress if it is possible at all. We challenge this assumption by arguing that if Global Workspace Theory (GWT) — a leading scientific theory of phenomenal consciousness — is correct, then instances of one widely implemented AI architecture, the artificial language agent, might easily be made phenomenally conscious if they are not already. Along the way, (...) we articulate an explicit methodology for thinking about how to apply scientific theories of consciousness to artificial systems and employ this methodology to arrive at a set of necessary and sufficient conditions for phenomenal consciousness according to GWT. (shrink)

The Great Divide in metaphysical debates about laws of nature is between Humeans, who think that laws merely describe the distribution of matter, and non-Humeans, who think that laws govern it. The metaphysics can place demands on the proper formulations of physical theories. It is sometimes assumed that the governing view requires a fundamental / intrinsic direction of time: to govern, laws must be dynamical, producing later states of the world from earlier ones, in accord with the fundamental direction of (...) time in the universe. In this paper, we propose a minimal primitivism about laws of nature (MinP) according to which there is no such requirement. On our view, laws govern by constraining the physical possibilities. Our view captures the essence of the governing view without taking on extraneous commitments about the direction of time or dynamic production. Moreover, as a version of primitivism, our view requires no reduction / analysis of laws in terms of universals, powers, or dispositions. Our view accommodates several potential candidates for fundamental laws, including the principle of least action, the Past Hypothesis, the Einstein equation of general relativity, and even controversial examples found in the Wheeler-Feynman theory of electrodynamics and retrocausal theories of quantum mechanics. By understanding governing as constraining, non-Humeans who accept MinP have the same freedom to contemplate a wide variety of candidate fundamental laws as Humeans do. (shrink)

This paper develops an information-sensitive theory of the semantics and probability of conditionals and statements involving epistemic modals. The theory validates a number of principles linking probability and modality, including the principle that the probability of a conditional If A, then C equals the probability of C, updated with A. The theory avoids so-called triviality results, which are standardly taken to show that principles of this sort cannot be validated. To achieve this, we deny that rational agents update their credences (...) via conditionalization. We offer a new rule of update, Hyperconditionalization, which agrees with Conditionalization whenever nonmodal statements are at stake but differs for modal and conditional sentences. (shrink)

Recent advances in natural language processing have given rise to a new kind of AI architecture: the language agent. By repeatedly calling an LLM to perform a variety of cognitive tasks, language agents are able to function autonomously to pursue goals specified in natural language and stored in a human-readable format. Because of their architecture, language agents exhibit behavior that is predictable according to the laws of folk psychology: they function as though they have desires and beliefs, and then make (...) and update plans to pursue their desires given their beliefs. We argue that the rise of language agents significantly reduces the probability of an existential catastrophe due to loss of control over an AGI. This is because the probability of such an existential catastrophe is proportional to the difficulty of aligning AGI systems, and language agents significantly reduce that difficulty. In particular, language agents help to resolve three important issues related to aligning AIs: reward misspecification, goal misgeneralization, and uninterpretability. (shrink)

Humans and animals make inferences about the world under limited time and knowledge. In contrast, many models of rational inference treat the mind as a Laplacean Demon, equipped with unlimited time, knowledge, and computational might. Following H. Simon's notion of satisficing, the authors have proposed a family of algorithms based on a simple psychological mechanism: one-reason decision making. These fast and frugal algorithms violate fundamental tenets of classical rationality: They neither look up nor integrate all information. By computer simulation, the (...) authors held a competition between the satisficing "Take The Best" algorithm and various "rational" inference procedures. The Take The Best algorithm matched or outperformed all competitors in inferential speed and accuracy. This result is an existence proof that cognitive mechanisms capable of successful performance in the real world do not need to satisfy the classical norms of rational inference. (shrink)

Recent research has identified a tension between the Safety principle that knowledge is belief without risk of error, and the Closure principle that knowledge is preserved by competent deduction. Timothy Williamson reconciles Safety and Closure by proposing that when an agent deduces a conclusion from some premises, the agent’s method for believing the conclusion includes their method for believing each premise. We argue that this theory is untenable because it implies problematically easy epistemic access to one’s methods. Several possible solutions (...) are explored and rejected. (shrink)

Formal models of appearance and reality have proved fruitful for investigating structural properties of perceptual knowledge. This paper applies the same approach to epistemic justification. Our central goal is to give a simple account of The Preface, in which justified belief fails to agglomerate. Following recent work by a number of authors, we understand knowledge in terms of normality. An agent knows p iff p is true throughout all relevant normal worlds. To model The Preface, we appeal to the normality (...) of error. Sometimes, it is more normal for reality and appearance to diverge than to match. We show that this simple idea has dramatic consequences for the theory of knowledge and justification. Among other things, we argue that a proper treatment of The Preface requires a departure from the internalist idea that epistemic justification supervenes on the appearances and the widespread idea that one knows most when free from error. (shrink)

Bohmian mechanics and the Ghirardi-Rimini-Weber theory provide opposite resolutions of the quantum measurement problem: the former postulates additional variables (the particle positions) besides the wave function, whereas the latter implements spontaneous collapses of the wave function by a nonlinear and stochastic modification of Schrödinger's equation. Still, both theories, when understood appropriately, share the following structure: They are ultimately not about wave functions but about 'matter' moving in space, represented by either particle trajectories, fields on space-time, or a discrete set of (...) space-time points. The role of the wave function then is to govern the motion of the matter. (shrink)

This paper offers a new argument against the KK thesis, which says that if you know p, then you know that you know p. We argue that KK is inconsistent with the fact that anyone denies the KK thesis: imagine that Dudley says he knows p but that he does not have 100 iterations of knowledge about p. If KK were true, Dudley would know that he has 100 iterations of knowledge about p, and so he wouldn’t deny that he (...) did. We consider several epicycles, and also explore whether the argument type also challenges other structural conditions on knowledge, such as closure under deduction. (shrink)

This paper explores the principle that knowledge is fragile, in that whenever S knows that S doesn’t know that S knows that p, S thereby fails to know p. Fragility is motivated by the infelicity of dubious assertions, utterances which assert p while acknowledging higher-order ignorance whether p. Fragility is interestingly weaker than KK, the principle that if S knows p, then S knows that S knows p. Existing theories of knowledge which deny KK by accepting a Margin for Error (...) principle can be conservatively extended with Fragility. (shrink)

Bohmian mechanics, which is also called the de Broglie-Bohm theory, the pilot-wave model, and the causal interpretation of quantum mechanics, is a version of quantum theory discovered by Louis de Broglie in 1927 and rediscovered by David Bohm in 1952. It is the simplest example of what is often called a hidden variables interpretation of quantum mechanics. In Bohmian mechanics a system of particles is described in part by its wave function, evolving, as usual, according to Schrödinger's equation. However, the (...) wave function provides only a partial description of the system. This description is completed by the specification of the actual positions of the particles. The latter evolve according to the.. (shrink)

AI companies are racing to create artificial general intelligence, or “AGI.” If they succeed, the result will be human-level AI systems that can independently pursue high-level goals by formulating and executing long-term plans in the real world. Leading AI researchers agree that some of these systems will likely be “misaligned”–pursuing goals that humans do not desire. This goal mismatch will put misaligned AIs and humans into strategic competition with one another. As with present-day strategic competition between nations with incompatible goals, (...) the result could be violent and catastrophic conflict. Existing legal institutions are unprepared for the AGI world. New foundations for AGI governance are needed, and the time to begin laying them is now, before the critical moment arrives. -/- This Article begins to lay those new legal foundations. It is the first to think systematically about the dynamics of strategic competition between humans and misaligned AGI. The Article begins by showing, using formal game-theoretic models, that, by default, humans and AIs will be trapped in a prisoner’s dilemma. Both parties’ dominant strategy will be to permanently disempower or destroy the other, even though the costs of such conflict would be high. -/- The Article then argues that a surprising legal intervention could transform the game theoretic equilibrium and avoid conflict: AI rights. Not just any AI rights would promote human safety. Granting AIs the right not to be needlessly harmed–as humans have granted to certain non-human animals–would, for example, have little effect. Instead, to promote human safety, AIs should be given those basic private law rights–to make contracts, hold property, and bring tort claims–that law already extends to non-human corporations. Granting AIs these economic rights would enable long-run, small-scale, mutually-beneficial transactions between humans and AIs. This would, we show, facilitate a peaceful strategic equilibrium between humans and AIs for the same reasons economic interdependence tends to promote peace in international relations. Namely, the gains from trade far exceed those from war. Throughout, we argue that human safety, rather than AI welfare, provides the right framework for developing AI rights. This Article explores both the promise and the limits of AI rights as a legal tool for promoting human safety in an AGI world. (shrink)

You omega know p when you possess every iteration of knowledge of p. This book argues that omega knowledge plays a central role in philosophy. In particular, the book argues that omega knowledge is necessary for permissible assertion, action, inquiry, and belief. Although omega knowledge plays this important role, existing theories of omega knowledge are unsatisfying. One theory, KK, identifies knowledge with omega knowledge. This theory struggles to accommodate cases of inexact knowledge. The other main theory is skeptical, claiming that (...) we do not omega know any ordinary claims about the world. This book develops and critically compares three new theories of omega knowledge. (shrink)

Foreword by Oliver Sacks Kurt Goldstein (1878-1965) was already an established neuropsychologist when he emigrated from Germany to the United States in the 1930s. This book, his magnum opus and widely regarded as a modern classic in psychology and biology, grew out of his dissatisfaction with traditional natural science techniques for analyzing living beings. It offers a broad introduction to the sources and ranges of application of the "holistic" or "organismic" research program that has since become a standard part of (...) biological thought. Goldstein was especially concerned with the breakdown of organization and the failure of central controls that take place in catastrophic responses to situations such as physical or mental illness. But he was equally attuned to the amazing powers of the organism to readjust to such catastrophic losses, if only by withdrawal to a more limited range that it could manage by a redistribution of its reduced energies, thus reclaiming as much wholeness as new circumstances allowed. Goldstein's theses in The Organism have had an important impact on philosophical and psychological thought throughout the twentieth century, as evidenced in the work of Maurice Merleau-Ponty, Georges Canguilhem, Ernst Cassirer, and Ludwig Binswanger. (shrink)

Philosophy and Phenomenological Research, EarlyView.

[Correction Notice: An erratum for this article was reported in Vol 109 of Psychological Review. Due to circumstances that were beyond the control of the authors, the studies reported in "Models of Ecological Rationality: The Recognition Heuristic," by Daniel G. Goldstein and Gerd Gigerenzer overlap with studies reported in "The Recognition Heuristic: How Ignorance Makes Us Smart," by the same authors and with studies reported in "Inference From Ignorance: The Recognition Heuristic". In addition, Figure 3 in the Psychological Review article (...) was originally published in the book chapter and should have carried a note saying that it was used by permission of Oxford University Press.] One view of heuristics is that they are imperfect versions of optimal statistical procedures considered too complicated for ordinary minds to carry out. In contrast, the authors consider heuristics to be adaptive strategies that evolved in tandem with fundamental psychological mechanisms. The recognition heuristic, arguably the most frugal of all heuristics, makes inferences from patterns of missing knowledge. This heuristic exploits a fundamental adaptation of many organisms: the vast, sensitive, and reliable capacity for recognition. The authors specify the conditions under which the recognition heuristic is successful and when it leads to the counter-intuitive less-is-more effect in which less knowledge is better than more for making accurate inferences. (shrink)

This paper develops a question-sensitive theory of intention. We show that this theory explains some puzzling closure properties of intention. In particular, it can be used to explain why one is rationally required to intend the means to one’s ends, even though one is not rationally required to intend all the foreseen consequences of one’s intended actions. It also explains why rational intention is not always closed under logical implication, and why one can only intend outcomes that one believes to (...) be under one’s control. (shrink)

A major disagreement between different views about the foundations of quantum mechanics concerns whether for a theory to be intelligible as a fundamental physical theory it must involve a ‘primitive ontology’ (PO), i.e. variables describing the distribution of matter in four-dimensional space–time. In this article, we illustrate the value of having a PO. We do so by focusing on the role that the PO plays for extracting predictions from a given theory and discuss valid and invalid derivations of predictions. To (...) this end, we investigate a number of examples based on toy models built from the elements of familiar interpretations of quantum theory. (shrink)

In the last quarter of the nineteenth century, Ludwig Boltzmann explained how irreversible macroscopic laws, in particular the second law of thermodynamics, originate in the time-reversible laws of microscopic physics. Boltzmann’s analysis, the essence of which I shall review here, is basically correct. The most famous criticisms of Boltzmann’s later work on the subject have little merit. Most twentieth century innovations – such as the identification of the state of a physical system with a probability distribution on its phase space, (...) of its thermodynamic entropy with the Gibbs entropy of , and the invocation of the notions of ergodicity and mixing for the justification of the foundations of statistical mechanics – are thoroughly misguided. (shrink)

We often claim to know what might be—or probably is—the case. Modal knowledge along these lines creates a puzzle for information-sensitive semantics for epistemic modals. This paper develops a solution. We start with the idea that knowledge requires safe belief: a belief amounts to knowledge only if it could not easily have been held falsely. We then develop an interpretation of the modal operator in safety that allows it to non-trivially embed information-sensitive contents. The resulting theory avoids various paradoxes that (...) arise from other accounts of modal knowledge. It also delivers plausible predictions about modal Gettier cases. (shrink)

What is it to believe something might be the case? We develop a puzzle that creates difficulties for standard answers to this question. We go on to propose our own solution, which integrates a Bayesian approach to belief with a dynamic semantics for epistemic modals. After showing how our account solves the puzzle, we explore a surprising consequence: virtually all of our beliefs about what might be the case provide counterexamples to the view that rational belief is closed under logical (...) implication. (shrink)

Quantum philosophy, a peculiar twentieth-century malady, is responsible for most of the conceptual muddle plaguing the foundations of quantum physics. When this philosophy is eschewed, one naturally arrives at Bohmian mechanics, which is what emerges from Schrodinger's equation for a nonrelativistic system of particles when we merely insist that 'particles' means particles. While distinctly non-Newtonian, Bohmian mechanics is a fully deterministic theory of particles in motion, a motion choreographed by the wave function. The quantum formalism emerges when measurement situations are (...) analyzed according to this theory. When the quantum formalism is regarded as arising in this way, the paradoxes and perplexities so often associated with quantum theory simply evaporate.Bohr's ... approach to atomic problems ... is really remarkable. He is completely convinced that any understanding in the usual sense of the word is impossible. Therefore the conversation is almost immediately driven into philosophical questions, and soon you no longer know whether you really take the position he is attacking, or whether you really must attack the position he is defending. (shrink)

That all pleasure is good and all pain bad in itself is an eternally true ethical principle. The common claim that some pleasure is not good, or some pain not bad, is mistaken. Strict particularism (ethical decisions must be made case by case; there are no sound universal normative principles) and relativism (all good and bad are relative to society) are among the ethical theories we may refute through an appeal to pleasure and pain. Daniel Dennett, Philippa Foot, R M (...) Hare, Gilbert Harman, Immanuel Kant, J. L. Mackie, and Jean-Paul Sartre are among the many philosophers addressed. (shrink)

You omega know something when you know it, and know that you know it, and know that you know that you know it, and so on. This paper first argues that omega knowledge matters, in the sense that it is required for rational assertion, action, inquiry, and belief. The paper argues that existing accounts of omega knowledge face major challenges. One account is skeptical, claiming that we have no omega knowledge of any ordinary claims about the world. Another account embraces (...) the KK thesis, and identifies knowledge with omega knowledge. This position faces counterexamples, and struggles to make sense of inexact knowledge. The paper then develops a new account of knowledge, by proposing the principle of Reflective Luminosity: if you know that you know something, then you omega know it. I argue that Reflective Luminosity allows for omega knowledge while avoiding the problems for KK. (shrink)

Schrödinger’s first proposal for the interpretation of quantum mechanics was based on a postulate relating the wave function on configuration space to charge density in physical space. Schrödinger apparently later thought that his proposal was empirically wrong. We argue here that this is not the case, at least for a very similar proposal with charge density replaced by mass density. We argue that when analyzed carefully, this theory is seen to be an empirically adequate many-worlds theory and not an empirically (...) inadequate theory describing a single world. Moreover, this formulation—Schrödinger’s first quantum theory—can be regarded as a formulation of the many-worlds view of quantum mechanics that is ontologically clearer than Everett’s. (shrink)

Many defend the thesis that when someone knows p, they couldn’t easily have been wrong about p. But the notion of easy possibility in play is relatively undertheorized. One structural idea in the literature, the principle of Counterfactual Closure (CC), connects easy possibility with counterfactuals: if it easily could have happened that p, and if p were the case, then q would be the case, it follows that it easily could have happened that q. We first argue that while CC (...) is false, there is a true restriction of it to cases involving counterfactual dependence on a coin flip. The failure of CC falsifies a model where the easy possibilities are counterfactually similar to actuality. Next, we show that extant normality models, where the easy possibilities are the sufficiently normal ones, are incompatible with the restricted CC thesis involving coin flips. Next, we develop a new kind of normality theory that can accommodate the restricted version of CC. This new theory introduces a principle of Counterfactual Contamination, which says roughly that any world is fairly abnormal if at that world very abnormal events counterfactually depend on a coin flip. Finally, we explain why coin flips and other related events have a special status. A central take home lesson is that the correct principle in the vicinity of Safety is importantly normality-theoretic rather than (as it is usually conceived) similarity-theoretic. (shrink)

Bohmian mechanics is arguably the most naively obvious embedding imaginable of Schr¨ odinger’s equation into a completely coherent physical theory. It describes a world in which particles move in a highly non-Newtonian sort of way, one which may at first appear to have little to do with the spectrum of predictions of quantum mechanics. It turns out, however, that as a consequence of the defining dynamical equations of Bohmian mechanics, when a system has wave function ψ its configuration is typically (...) random, with probability density ρ given by |ψ|2, the quantum equilibrium distribution. It also turns out that the entire quantum formalism, operators as observables and all the rest, naturally emerges in Bohmian mechanics from the analysis of “measurements.” This analysis reveals the status of operators as observables in the description of quantum phenomena, and facilitates a clear view of the range of applicability of the usual quantum mechanical formulas. (shrink)

This paper argues that a range of current AI systems have learned how to deceive humans. We define deception as the systematic inducement of false beliefs in the pursuit of some outcome other than the truth. We first survey empirical examples of AI deception, discussing both special-use AI systems (including Meta's CICERO) built for specific competitive situations, and general-purpose AI systems (such as large language models). Next, we detail several risks from AI deception, such as fraud, election tampering, and losing (...) control of AI systems. Finally, we outline several potential solutions to the problems posed by AI deception: first, regulatory frameworks should subject AI systems that are capable of deception to robust risk-assessment requirements; second, policymakers should implement bot-or-not laws; and finally, policymakers should prioritize the funding of relevant research, including tools to detect AI deception and to make AI systems less deceptive. Policymakers, researchers, and the broader public should work proactively to prevent AI deception from destabilizing the shared foundations of our society. (shrink)

This paper offers the first careful analysis of the possibility that AI and humanity will go to war. The paper focuses on the case of artificial general intelligence, AI with broadly human capabilities. The paper uses a bargaining model of war to apply standard causes of war to the special case of AI/human conflict. The paper argues that information failures and commitment problems are especially likely in AI/human conflict. Information failures would be driven by the difficulty of measuring AI capabilities, (...) by the uninterpretability of AI systems, and by differences in how AIs and humans analyze information. Commitment problems would make it difficult for AIs and humans to strike credible bargains. Commitment problems could arise from power shifts, rapid and discontinuous increases in AI capabilities. Commitment problems could also arise from missing focal points, where AIs and humans fail to effectively coordinate on policies to limit war. In the face of this heightened chance of war, the paper proposes several interventions. War can be made less likely by improving the measurement of AI capabilities, capping improvements in AI capabilities, designing AI systems to be similar to humans, and by allowing AI systems to participate in democratic political institutions. (shrink)

The most puzzling issue in the foundations of quantum mechanics is perhaps that of the status of the wave function of a system in a quantum universe. Is the wave function objective or subjective? Does it represent the physical state of the system or merely our information about the system? And if the former, does it provide a complete description of the system or only a partial description? We shall address these questions here mainly from a Bohmian perspective, and shall (...) argue that part of the difficulty in ascertaining the status of the wave function in quantum mechanics arises from the fact that there are two different sorts of wave functions involved. The most fundamental wave function is that of the universe. From it, together with the configuration of the universe, one can define the wave function of a subsystem. We argue that the fundamental wave function, the wave function of the universe, has a law-like character. (shrink)

This paper develops a semantic solution to the puzzle of Free Choice permission. The paper begins with a battery of impossibility results showing that Free Choice is in tension with a variety of classical principles, including Disjunction Introduction and the Law of Excluded Middle. Most interestingly, Free Choice appears incompatible with a principle concerning the behavior of Free Choice under negation, Double Prohibition, which says that Mary can’t have soup or salad implies Mary can’t have soup and Mary can’t have (...) salad. Alonso-Ovalle 2006 and others have appealed to Double Prohibition to motivate pragmatic accounts of Free Choice. Aher 2012, Aloni 2018, and others have developed semantic accounts of Free Choice that also explain Double Prohibition. -/- This paper offers a new semantic analysis of Free Choice designed to handle the full range of impossibility results involved in Free Choice. The paper develops the hypothesis that Free Choice is a homogeneity effect. The claim possibly A or B is defined only when A and B are homogenous with respect to their modal status, either both possible or both impossible. Paired with a notion of entailment that is sensitive to definedness conditions, this theory validates Free Choice while retaining a wide variety of classical principles except for the transitivity of entailment. The homogeneity hypothesis is implemented in two different ways, homogeneous alternative semantics and homogeneous dynamic semantics, with interestingly different consequences. (shrink)

This paper models knowledge in cases where an agent has multiple experiences over time. Using this model, we introduce a series of observations that undermine the pretheoretic idea that the evidential significance of experience depends on the extent to which that experience matches the world. On the basis of these observations, we model knowledge in terms of what is likely given the agent’s experience. An agent knows p when p is implied by her epistemic possibilities. A world is epistemically possible (...) when its probability given the agent’s experiences is not significantly lower than the probability of the actual world given that experience. (shrink)

We propose developing AIs whose only final goal is being shut down. We argue that this approach to AI safety has three benefits: (i) it could potentially be implemented in reinforcement learning, (ii) it avoids some dangerous instrumental convergence dynamics, and (iii) it creates trip wires for monitoring dangerous capabilities. We also argue that the proposal can overcome a key challenge raised by Soares et al. (2015), that shutdown-seeking AIs will manipulate humans into shutting them down. We conclude by comparing (...) our approach with Soares et al.'s corrigibility framework. (shrink)

LLMs have dramatically improved in capabilities in recent years. This raises the question of whether LLMs could become genuine agents with beliefs and desires. This paper demonstrates an in principle limit to LLM agency, based on their architecture. LLMs are next word predictors: given a string of text, they calculate the probability that various words can come next. LLMs produce outputs that reflect these probabilities. I show that next word predictors are exploitable. If LLMs are prompted to make probabilistic predictions (...) about the world, these predictions are guaranteed to be incoherent, and so Dutch bookable. If LLMs are prompted to make choices over actions, their preferences are guaranteed to be intransitive, and so money pumpable. In short, the problem is that selecting an action based on its potential value is structurally different then selecting the description of an action that is most likely given a prompt: probability cannot be forced into the shape of expected value. The in principle exploitability of LLMs raises doubts about how agential they can become. This exploitability also offers an opportunity for humanity to safely control such AI systems. (shrink)

We are often told that quantum phenomena demand radical revisions of our scientific world view and that no physical theory describing well defined objects, such as particles described by their positions, evolving in a well defined way, let alone deterministically, can account for such phenomena. The great majority of physicists continue to subscribe to this view, despite the fact that just such a deterministic theory, accounting for all of the phe nomena of nonrelativistic quantum mechanics, was proposed by David Bohm (...) more than four decades ago and has arguably been around almost since the inception of quantum mechanics itself. Our purpose in asking colleagues to write the essays for this volume has not been to produce a Festschrift in honor of David Bohm or to gather together a collection of papers simply stating uncritically Bohm's views on quantum mechanics. The central theme around which the essays in this volume are arranged is David Bohm's version of quantum mechanics. It has by now become fairly standard practice to refer to his theory as Bohmian mechanics and to the larger conceptual framework within which this is located as the causal quantum theory program. While it is true that one can have reservations about the appropriateness of these specific labels, both do elicit distinc tive images characteristic of the key concepts of these approaches and such terminology does serve effectively to contrast this class of theories with more standard formulations of quantum theory. (shrink)

Triviality results threaten plausible principles governing our credence in epistemic modal claims. This paper develops a new account of modal credence which avoids triviality. On the resulting theory, probabilities are assigned not to sets of worlds, but rather to sets of information state-world pairs. The theory avoids triviality by giving up the principle that rational credence is closed under conditionalization. A rational agent can become irrational by conditionalizing on new evidence. In place of conditionalization, the paper develops a new account (...) of updating: conditionalization with normalization. (shrink)

This paper explores the relationship between dynamic and truth conditional semantics for epistemic modals. It provides a generalization of a standard dynamic update semantics for modals. This new semantics derives a Kripke semantics for modals and a standard dynamic semantics for modals as special cases. The semantics allows for new characterizations of a variety of principles in modal logic, including the inconsistency of ‘p and might not p’. Finally, the semantics provides a construction procedure for transforming any truth conditional semantics (...) for modals into a dynamic semantics for modals with similar properties. (shrink)

Against Hume and Epicurus I argue that our selection of pleasure, pain and other objects as our ultimate ends is guided by reason. There are two parts to the explanation of our attraction to pleasure, our aversion to pain, and our consequent preference of pleasure to pain: 1. Pleasure presents us with reason to seek it, pain presents us reason to avoid it, and 2. Being intelligent, human beings (and to a degree, many animals) are disposed to be guided by (...) reason, and hence by what there is reason to choose, seek, and prefer, when they act. (shrink)

Recent philosophical literature on epistemic harms has paid little attention to the difference between deliberate and non-deliberate harms. In this paper, I analyze the “Curare Case,” a case from the 1940’s in which patient testimony was disregarded by physicians. This case has been described as an instance of epistemic injustice. I problematize this description, arguing instead that the case shows an instance of “epistemic disadvantage.” I propose epistemic disadvantage indicates when harms result from warranted asymmetric relations that justifiably exclude individuals (...) from hermeneutical participation. Epistemic disadvantage categorizes harms that result from justifiable exclusions, are non-deliberate, and result from poor epistemic environments. This analysis brings out a meaningful difference between accidental and deliberate harms in communicative exchanges. (shrink)

In this paper I argue that there is a preface paradox for intention. The preface paradox for intention shows that intentions do not obey an agglomeration norm, requiring one to intend conjunctions of whatever else one intends. But what norms do intentions obey? I will argue that intentions come in degrees. These partial intentions are governed by the norms of the probability calculus. First, I will give a dispositional theory of partial intention, on which degrees of intention are the degrees (...) to which one possesses the dispositions characteristic of full intention. I will use this dispositional theory to defend probabilism about intention. Next, I will offer a more general argument for probabilism about intention. To do so, I will generalize recent decision theoretic arguments for probabilism from the case of belief to the case of intention. (shrink)

As participatory research practices are increasingly taken up in health research, claims related to experiential authority and expertise are frequently made. Here, in an exploration of what grounds such claims, we consider how feminist standpoint theory might apply to the psy sciences (psychiatry, psychology, psychotherapy, psychoanalysis, and so on). Standpoint theory claims that experiences of marginalization and critical engagement can lead to a standpoint that offers an epistemic advantage within a domain of knowledge. We examine experiences of marginalization and critical (...) engagement in the mental health system, as well as evidence for epistemic advantages resulting from these experiences. This evidence, found in the identification of problematic assumptions and the development of new tools and theories in the field, grounds our argument that standpoint theory is indeed relevant to the psy sciences and that many of those who have experienced marginalization and have engaged critically have an epistemic advantage when it comes to knowledge-production. The implications of this argument are significant: those who have attained a standpoint within the psy sciences ought to be included in research and given both tools and funding to develop research programs. However, we must be wary of the risks of tokenization, cooptation, and essentialization that are likely to accompany such a transformation. (shrink)