The Turing Test is one of the most disputed topics in artificial intelligence, philosophy of mind, and cognitive science. This paper is a review of the past 50 years of the Turing Test. Philosophical debates, practical developments and repercussions in related disciplines are all covered. We discuss Turing's ideas in detail and present the important comments that have been made on them. Within this context, behaviorism, consciousness, the 'other minds' problem, and similar topics in philosophy (...) of mind are discussed. We also cover the sociological and psychological aspects of the Turing Test. Finally, we look at the current situation and analyze programs that have been developed with the aim of passing the Turing Test. We conclude that the Turing Test has been, and will continue to be, an influential and controversial topic. -/- (This paper was reprinted in: The Turing Test: The Elusive Standard of Artificial Intelligence, J. H. Moor, ed., Kluwer Academic, pp. 23-78, 2003.). (shrink)

The Turing test, which is a verbal test of the indistinguishability of machine and human intelligence, is a historically important idea that has set a way of thinking about the AI (artificial intelligence) project that is still relevant today. According to it, the benchmark/blueprint for AI is human intelligence, and the key skill of AI should be its communicative proficiency – which includes explaining decisions made by the machine. Passing the original Turing test by a (...) machine does not guarantee that the machine is, from the point of view of a human, trustworthy; on the contrary, the idea of a machine capable of “fooling” or “outsmarting” a human is inherent within the concept of the test. We postulate, that a test that guarantees a high level of trust should be: a) non-imitative, b) non-behavioral, c) focused on the ability to explain, taking into account, however, the design and rules of operation of the machine (and not just human expectations), d) focused on the machine’s ability to learn. (shrink)

The Turing test has been studied and run as a controlled experiment and found to be underspecified and poorly designed. On the other hand, it has been defended and still attracts interest as a test for true artificial intelligence (AI). Scientists and philosophers regret the test’s current status, acknowledging that the situation is at odds with the intellectual standards of Turing’s works. This article refers to this as the Turing Test Dilemma, following the (...) observation that the test has been under discussion for over seventy years and still is widely seen as either too bad or too good to be a valuable experiment for AI. An argument that solves the dilemma is presented, which relies on reconstructing the Turing test as a thought experiment in the modern scientific tradition. It is argued that Turing’s exposition of the imitation game satisfies Mach’s characterization of the basic method of thought experiments and that Turing’s uses of his test satisfy Popper’s conception of the critical and heuristic uses of thought experiments and Kuhn’s association of thought experiments to conceptual change. It is emphasized how Turing methodically varied the imitation game design to address specific challenges posed to him by other thinkers and how his test illustrates a property of the phenomenon of intelligence and suggests a hypothesis on machine learning. This reconstruction of the Turing test provides a rapprochement to the conflicting views on its value in the literature. (shrink)

Discussions of artificial intelligence have been shaped by two brilliant thought-experiments: Turing’s Imitation Test for thinking systems and Searle’s Chinese Room Argument. In many ways, debates about large language models (LLMs) struggle to move beyond these original, opposing thought-experiments. So, in this paper, I ask whether we can move debate forward by exploring the features Sceptics about LLM abilities take to ground meaning. Section 1 sketches the options, while Sections 2 and 3 explore the common requirement for a (...) robust relation between linguistic signs and external objects. Section 3 argues that concerns about worldly connections can be met and thus that outputs of LLMs should be viewed as genuinely meaningful. Section 4 then turns to the argument that the kind of derived meaning explored in Section 3 is insufficient for LLMs to count as meaningful systems per se, looking at the claim that they must possess original intentionality before we admit them to the space of meaning. I argue that this demand is not a prerequisite for meaning per se, but rather for some kinds of agency or conscious understanding. I suggest that the demands for original intentionality are not currently met by LLMs and that we should be cautious about whether we want them to be. (shrink)

Turing''s test has been much misunderstood. Recently unpublished material by Turing casts fresh light on his thinking and dispels a number of philosophical myths concerning the Turing test. Properly understood, the Turing test withstands objections that are popularly believed to be fatal.

Stuart M. Shieber’s name is well known to computational linguists for his research and to computer scientists more generally for his debate on the Loebner Turing Test competition, which appeared a decade earlier in Communications of the ACM. 1 With this collection, I expect it to become equally well known to philosophers.

This paper provides a survey of philosophical discussion of the "the Turing Test". In particular, it provides a very careful and thorough discussion of the famous 1950 paper that was published in Mind.

In 1950, Alan Turing proposed his eponymous test based on indistinguishability of verbal behavior as a replacement for the question "Can machines think?" Since then, two mutually contradictory but well-founded attitudes towards the Turing Test have arisen in the philosophical literature. On the one hand is the attitude that has become philosophical conventional wisdom, viz., that the Turing Test is hopelessly flawed as a sufficient condition for intelligence, while on the other hand is the (...) overwhelming sense that were a machine to pass a real live full-fledged Turing Test, it would be a sign of nothing but our orneriness to deny it the attribution of intelligence. The arguments against the sufficiency of the Turing Test for determining intelligence rely on showing that some extra conditions are logically necessary for intelligence beyond the behavioral properties exhibited by an agent under a Turing Test. Therefore, it cannot follow logically from passing a Turing Test that the agent is intelligent. I argue that these extra conditions can be revealed by the Turing Test, so long as we allow a very slight weakening of the criterion from one of logical proof to one of statistical proof under weak realizability assumptions. The argument depends on the notion of interactive proof developed in theoretical computer science, along with some simple physical facts that constrain the information capacity of agents. Crucially, the weakening is so slight as to make no conceivable difference from a practical standpoint. Thus, the Gordian knot between the two opposing views of the sufficiency of the Turing Test can be cut. (shrink)

The Turing Test, originally proposed as a simple operational definition of intelligence, has now been with us for exactly half a century. It is safe to say that no other single article in computer science, and few other articles in science in general, have generated so much discussion. The present article chronicles the comments and controversy surrounding Turing's classic article from its publication to the present. The changing perception of the Turing Test over the last (...) fifty years has paralleled the changing attitudes in the scientific community towards artificial intelligence: from the unbridled optimism of 1960's to the current realization of the immense difficulties that still lie ahead. I conclude with the prediction that the Turing Test will remain important, not only as a landmark in the history of the development of intelligent machines, but also with real relevance to future generations of people living in a world in which the cognitive capacities of machines will be vastly greater than they are now. (shrink)

There has been considerable optimistic speculation on how well ChatGPT-4 would perform in a Turing Test. However, no minimally serious implementation of the test has been reported to have been carried out. This brief note documents the re-sults of subjecting ChatGPT-4 to 10 Turing Tests, with different interrogators and participants. The outcome is tremendously disappointing for the optimists. Despite ChatGPT reportedly outperforming 99.9% of humans in a Verbal IQ test, it falls short of passing the (...) Turing Test. In 9 out of the 10 tests conducted, the interroga-tors successfully identified ChatGPT-4 and the human participant. The probability of obtaining this result from a process in which the interrogator is really no better than chance at correct identification is calculated to be less than 1%. An additional question was posed to the interrogators at the end of each test: What led them to distinguish between the human and the machine? The interrogators, who effectively filtered out ChatGPT-4 from passing the Turing Test for intelligence, stated that they could identify the machine because it, in effect, responded more intelligently than the human. Subsequently, ChatGPT-4 was tasked with differentiating syntax from semantics and self-corrected when falling for the fallacy of equivocation. The curious situation is arrived at that passing the Turing Test for intelligence remains a challenge that ChatGPT-4 has yet to overcome, precisely because, as per the inter-rogators, its intellectual abilities surpass those of individual humans. (shrink)

I have discussed the frame problem and the Turing test at length, but I have not attempted to spell out what I think the implications of the frame problem ...

In the classical Turing test, participants are challenged to tell whether they are interacting with another human being or with a machine. The way the interaction takes place is not direct, but a distant conversation through computer screen messages. Basic forms of interaction are face-to-face and embodied, context-dependent and based on the detection of reciprocal sensorimotor contingencies. Our idea is that interaction detection requires the integration of proprioceptive and interoceptive patterns with sensorimotor patterns, within quite short time lapses, (...) so that they appear as mutually contingent, as reciprocal. In other words, the experience of interaction takes place when sensorimotor patterns are contingent upon one’s own movements, and vice versa. I react to your movement, you react to mine. When I notice both components, I come to experience an interaction. Therefore, we designed a “minimal” Turing test to investigate how much information is required to detect these reciprocal sensorimotor contingencies. Using a new version of the perceptual crossing paradigm, we tested whether participants resorted to interaction detection to tell apart human from machine agents in repeated encounters with these agents. In two studies, we presented participants with movements of a human agent, either online or offline, and movements of a computerized oscillatory agent in three different blocks. In each block, either auditory or audiovisual feedback was provided along each trial. Analysis of participants’ explicit responses and of the implicit information subsumed in the dynamics of their series will reveal evidence that participants use the reciprocal sensorimotor contingencies within short time windows. For a machine to pass this minimal Turing test, it should be able to generate this sort of reciprocal contingencies. (shrink)

Brain damage can cause massive changes in consciousness levels. From a clinical and ethical point of view it is desirable to assess the level of residual consciousness in unresponsive patients. However, no direct measure of consciousness exists, so we run into the philosophical problem of other minds. Neurologists often make implicit use of a Turing test-like procedure in an attempt to gain access to damaged minds, by monitoring and interpreting neurobehavioral responses. New brain imaging techniques are now being (...) developed that permit communication with unresponsive patients, using their brain signals as carriers of messages relating to their mental states. (shrink)

The Turing Test is claimed by many to be a way to test for the presence, in computers, of such ``deep'' phenomena as thought and consciousness. Unfortunately, attempts to build computational systems able to pass TT have devolved into shallow symbol manipulation designed to, by hook or by crook, trick. The human creators of such systems know all too well that they have merely tried to fool those people who interact with their systems into believing that these (...) systems really have minds. And the problem is fundamental: the structure of the TT is such as to cultivate tricksters. A better test is one that insists on a certain restrictive epistemic relation between an artificial agent A, its output o, and the human architect H of A – a relation which, roughly speaking, obtains when H cannot account for how A produced o. We call this test the ``Lovelace Test'' in honor of Lady Lovelace, who believed that only when computers originate things should they be believed to have minds. (shrink)

This paper discusses some difficulties in understanding the Turing test. It emphasizes the importance of distinguishing between conceptual and empirical perspectives and highlights the former as introducing more serious problems for the TT. Some objections against the Turingian framework stemming from the later Wittgenstein’s philosophy are exposed. The following serious problems are examined: 1) It considers a unique and exclusive criterion for thinking which amounts to their identification; 2) it misidentifies the relationship of speaking to thinking as that (...) of a criterion; 3) it neglects the “natural” course of the development in semantics. However, these considerations suggest only that it is problematic to label a successful chatbot as a “thinking entity” without further qualifications, but not necessarily and once and for all incorrect. Philosophy has only little to say about the technical possibility of creating such an effective program. (shrink)

In this paper we discuss the views on the Turing test of four influential thinkers who belong to the tradition of analytic philosophy: Ludwig Wittgenstein, Noam Chomsky, Hilary Putnam and John Searle. Based on various beliefs about philosophical and/or linguistic matters, they arrive at different assessments of both the significance and suitability of the imitation game for the development of cognitive science and AI models. Nevertheless, they share a rejection of the idea that one can treat Turing (...) test as a test for “machine thinking.” This seems to stem from a concern for the proper use of language —one that is a fundamental methodological commitment of analytic philosophy. (shrink)

Alan Turing, einer der Gründerväter der modernen Informatik, diskutierte in seinem 1950 veröffentlichten Artikel »Computing Machinery and Intelligence« die Frage, ob Maschinen denken können. Dies wirft jedoch die schwierige Frage auf, was Denken denn sei. Um diese Frage zu umgehen, schlägt Turing vor, stattdessen eine Frage zu stellen, die sich durch bloße Beobachtung klären lässt, nämlich ob eine Maschine ein bestimmtes Spiel erfolgreich spielen könne.

The Turing Test is routinely understood as a behaviourist test for machine intelligence. Diane Proudfoot has argued for an alternative interpretation. According to Proudfoot, Turing’s claim that intelligence is what he calls ‘an emotional concept’ indicates that he conceived of intelligence in response-dependence terms. As she puts it: ‘Turing’s criterion for “thinking” is…: x is intelligent if in the actual world, in an unrestricted computer-imitates-human game, x appears intelligent to an average interrogator’. The role of (...) the famous test is thus to provide the conditions in which to examine the average interrogator’s responses. I shall argue that Proudfoot’s analysis falls short. The philosophical literature contains two main models of response-dependence, what I shall call the transparency model and the reference-fixing model. Proudfoot resists the thought that Turing might have endorsed one of these models to the exclusion of the other. But the details of her own analysis indicate that she is, in fact, committed to the claim that Turing’s account of intelligence is grounded in a transparency model, rather than a reference-fixing one. By contrast, I shall argue that while Turing did indeed conceive of intelligence in response-dependence terms, his account is grounded in a reference-fixing model, rather than a transparency one. This is fortunate, because, as an account of intelligence, the transparency model is arguably problematic in a way that the reference-fixing model isn’t. (shrink)

I advocate a theory of syntactic semantics as a way of understanding how computers can think (and how the Chinese-Room-Argument objection to the Turing Test can be overcome): (1) Semantics, considered as the study of relations between symbols and meanings, can be turned into syntax – a study of relations among symbols (including meanings) – and hence syntax (i.e., symbol manipulation) can suffice for the semantical enterprise (contra Searle). (2) Semantics, considered as the process of understanding one domain (...) (by modeling it) in terms of another, can be viewed recursively: The base case of semantic understanding –understanding a domain in terms of itself – is syntactic understanding. (3) An internal (or narrow ), first-person point of view makes an external (or wide ), third-person point of view otiose for purposes of understanding cognition. (shrink)

The Turing Test is best regarded as a model to test for intelligence, where an entity’s intelligence is inferred from its ability to be attributed with ‘human-likeness’ during a text-based conversation. The problem with this model, however, is that it does not care if or how well an entity produces a meaningful conversation, as long as its interactions are humanlike enough. As a consequence, the TT attracts projects that concentrate on how best to fool the judges. In (...) light of this, I propose a new version of the TT: the Questioning Turing Test. Here, the entity has to produce an enquiry rather than a conversation; and it is parametrised along two further dimensions in addition to ‘human-likeness’: ‘correctness’, evaluating if the entity accomplishes the enquiry; and ‘strategicness’, evaluating how well the entity accomplishes the enquiry, in terms of the number of questions asked. (shrink)

In 2014, widespread reports in the popular media that a chatbot named Eugene Goostman had passed the Turing test became further grist for those who argue that the diversionary tactics of chatbots like Goostman and others, such as those who participate in the Loebner competition, are enabled by the open-ended dialog of the Turing test. Some claim a new kind of test of machine intelligence is needed, and one community has advanced the Winograd schema competition (...) to address this gap. We argue to the contrary that implicit in the Turing test is the cooperative challenge of using language to build a practical working understanding, necessitating a human interrogator to monitor and direct the conversation. We give examples which show that, because ambiguity in language is ubiquitous, open-ended conversation is not a flaw but rather the core challenge of the Turing test. We outline a statistical notion of practical working understanding that permits a reasonable amount of ambiguity, but nevertheless requires that ambiguity be resolved sufficiently for the agents to make progress. (shrink)

This paper argues against the moral Turing test as a framework for evaluating the moral performance of autonomous systems. Though the term has been carefully introduced, considered, and cautioned about in previous discussions :251–261, 2000; Allen and Wallach 2009), it has lingered on as a touchstone for developing computational approaches to moral reasoning :98–109, 2015). While these efforts have not led to the detailed development of an MTT, they nonetheless retain the idea to discuss what kinds of action (...) and reasoning should be demanded of autonomous systems. We explore the flawed basis of an MTT in imitation, even one based on scenarios of morally accountable actions. MTT-based evaluations are vulnerable to deception, inadequate reasoning, and inferior moral performance vis a vis a system’s capabilities. We propose verification—which demands the design of transparent, accountable processes of reasoning that reliably prefigure the performance of autonomous systems—serves as a superior framework for both designer and system alike. As autonomous social robots in particular take on an increasing range of critical roles within society, we conclude that verification offers an essential, albeit challenging, moral measure of their design and performance. (shrink)

The main factor of intelligence is defined as the ability tocomprehend, formalising this ability with the help of new constructsbased on descriptional complexity. The result is a comprehension test,or C- test, which is exclusively defined in computational terms. Due toits absolute and non-anthropomorphic character, it is equally applicableto both humans and non-humans. Moreover, it correlates with classicalpsychometric tests, thus establishing the first firm connection betweeninformation theoretical notions and traditional IQ tests. The TuringTest is compared with the C- (...) class='Hi'>test and the combination of the two isquestioned. In consequence, the idea of using the Turing Test as apractical test of intelligence should be surpassed, and substituted bycomputational and factorial tests of different cognitive abilities, amuch more useful approach for artificial intelligence progress and formany other intriguing questions that present themselves beyond theTuring Test. (shrink)

The claim has often been made that passing the Turing Test would not be sufficient to prove that a computer program was intelligent because a trivial program could do it, namely, the “Humongous-Table (HT) Program”, which simply looks up in a table what to say next. This claim is examined in detail. Three ground rules are argued for: (1) That the HT program must be exhaustive, and not be based on some vaguely imagined set of tricks. (2) That (...) the HT program must not be created by some set of sentient beings enacting responses to all possible inputs. (3) That in the current state of cognitive science it must be an open possibility that a computational model of the human mind will be developed that accounts for at least its nonphenomenological properties. Given ground rule 3, the HT program could simply be an “optimized” version of some computational model of a mind, created via the automatic application of program-transformation rules [thus satisfying ground rule 2]. Therefore, whatever mental states one would be willing to impute to an ordinary computational model of the human psyche one should be willing to grant to the optimized version as well. Hence no one could dismiss out of hand the possibility that the HT program was intelligent. This conclusion is important because the Humongous-Table Program Argument is the only argument ever marshalled against the sufficiency of the Turing Test, if we exclude arguments that cognitive science is simply not possible. (shrink)

The so-called Turing test, as it is usually interpreted, sets a benchmark standard for determining when we might call a machine intelligent. We can call a machine intelligent if the following is satisfied: if a group of wise observers were conversing with a machine through an exchange of typed messages, those observers could not tell whether they were talking to a human being or to a machine. To pass the test, the machine has to be intelligent but (...) it also should be responsive in a manner which cannot be distinguished from a human being. This standard interpretation presents the Turing test as a criterion for demarcating intelligent from non-intelligent entities. For a long time proponents of artificial intelligence have taken the Turing test as a goalpost for measuring progress. (shrink)

Turing does not provide an explanation for substituting the original question of his test – i.e., “Can machines think?” with “Can a machine pass the imitation game?” – resulting in an argumentative gap in his main thesis. In this article, I argue that a positive answer to the second question would mean attributing the ability of linguistic interactions to machines; while a positive answer to the original question would mean attributing the ability of thinking to machines. In such (...) a situation, defending the Turing Test requires establishing a relationship between thought and language. In this regard, Davidson's no-priority theory is presented as an approach for defending the test. (shrink)

Anti-behaviorist arguments against the validity of the Turing Test as a sufficient condition for attributing intelligence are based on a memorizing machine, which has recorded within it responses to every possible Turing Test interaction of up to a fixed length. The mere possibility of such a machine is claimed to be enough to invalidate the Turing Test. I consider the nomological possibility of memorizing machines, and how long a Turing Test they can (...) pass. I replicate my previous analysis of this critical Turing Test length based on the age of the universe, show how considerations of communication time shorten that estimate and allow eliminating the sole remaining contingent assumption, and argue that the bound is so short that it is incompatible with the very notion of the Turing Test. I conclude that the memorizing machine objection to the Turing Test as a sufficient condition for attributing intelligence is invalid. (shrink)

The paper examines the nature of the behavioral evidence underlying attributions of intelligence in the case of human beings, and how this might be extended to other kinds of cognitive system, in the spirit of the original Turing Test. I consider Harnad's Total Turing Test, which involves successful performance of both linguistic and robotic behavior, and which is often thought to incorporate the very same range of empirical data that is available in the human case. However, (...) I argue that the TTT is still too weak, because it only tests the capabilities of particular tokens within a preexisting context of intelligent behavior. What is needed is a test of the cognitive type, as manifested through a number of exemplary tokens, in order to confirm that the cognitive type is able to produce the context of intelligent behavior presupposed by tests such as the TT and TTT. (shrink)

This article describes research to build an embodied conversational agent as an interface to a question-and-answer system about a National Science Foundation program. We call this ECA the LifeLike Avatar, and it can interact with its users in spoken natural language to answer general as well as specific questions about specific topics. In an idealized case, the LifeLike Avatar could conceivably provide a user with a level of interaction such that he or she would not be certain as to whether (...) he or she is talking to the actual person via video teleconference. This could be considered a extended version of the seminal Turing test. Although passing such a test is still far off, our work moves the science in that direction. The Uncanny Valley notwithstanding, applications of such lifelike interfaces could include those where specific instructors/caregivers could be represented as stand-ins for the actual person in situations where personal representation is important. Possible areas that come to mind that might benefit from these lifelike ECAs include health-care support for elderly/disabled patients in extended home care, education/training, and knowledge preservation. Another more personal application would be to posthumously preserve elements of the persona of a loved one by family members. We apply this approach to a Q/a system for knowledge preservation and dissemination, where the specific individual who had this knowledge was to retire from the US National Science Foundation. The system is described in detail, and evaluations were performed to determine how well the system was perceived by users. (shrink)

In this paper, I raise the question whether an artificial intelligence can act morally. I first sketch and defend a general picture of what is at stake in this question. I then sketch and defend a behavioral test, known as the Moral Turing Test, as a good sufficiency test for an artificial intelligence acting morally. I end by discussing some general anticipated objections.

In this paper, we look at the possibility of a machine having a sense of humour. In particular, we focus on actual machine utterances in Turing test discourses. In doing so, we do not consider the Turing test in depth and what this might mean for humanity, rather we merely look at cases in conversations when the output from a machine can be considered to be humorous. We link such outpourings with Turing’s “arguments from various (...) disabilities” used against the concept of a machine being able to think, taken from his seminal work of 1950. Finally we consider the role that humour might play in adding to the deception, integral to the Turing test, that a machine in practice appears to be a human. (shrink)

Advances in artificial intelligence (AI) raise important questions about whether people view moral evaluations by AI systems similarly to human-generated moral evaluations. We conducted a modified Moral Turing Test (m-MTT), inspired by Allen et al. (Exp Theor Artif Intell 352:24–28, 2004) proposal, by asking people to distinguish real human moral evaluations from those made by a popular advanced AI language model: GPT-4. A representative sample of 299 U.S. adults first rated the quality of moral evaluations when blinded to (...) their source. Remarkably, they rated the AI’s moral reasoning as superior in quality to humans’ along almost all dimensions, including virtuousness, intelligence, and trustworthiness, consistent with passing what Allen and colleagues call the comparative MTT. Next, when tasked with identifying the source of each evaluation (human or computer), people performed significantly above chance levels. Although the AI did not pass this test, this was not because of its inferior moral reasoning but, potentially, its perceived superiority, among other possible explanations. The emergence of language models capable of producing moral responses perceived as superior in quality to humans’ raises concerns that people may uncritically accept potentially harmful moral guidance from AI. This possibility highlights the need for safeguards around generative language models in matters of morality. (shrink)

Turing’s much debated test has turned 70 and is still fairly controversial. His 1950 paper is seen as a complex and multilayered text, and key questions about it remain largely unanswered. Why did Turing select learning from experience as the best approach to achieve machine intelligence? Why did he spend several years working with chess playing as a task to illustrate and test for machine intelligence only to trade it out for conversational question-answering in 1950? Why (...) did Turing refer to gender imitation in a test for machine intelligence? In this article, I shall address these questions by unveiling social, historical and epistemological roots of the so-called Turing test. I will draw attention to a historical fact that has been only scarcely observed in the secondary literature thus far, namely that Turing’s 1950 test emerged out of a controversy over the cognitive capabilities of digital computers, most notably out of debates with physicist and computer pioneer Douglas Hartree, chemist and philosopher Michael Polanyi, and neurosurgeon Geoffrey Jefferson. Seen in its historical context, Turing’s 1950 paper can be understood as essentially a reply to a series of challenges posed to him by these thinkers arguing against his view that machines can think. Turing did propose gender learning and imitation as one of his various imitation tests for machine intelligence, and I argue here that this was done in response to Jefferson's suggestion that gendered behavior is causally related to the physiology of sex hormones. (shrink)

The paper begins by examining the original Turing Test (2T) and Searle’s antithetical Chinese Room Argument, which is intended to refute the 2T in particular, as well as any formal or abstract procedural theory of the mind in general. In the ensuing dispute between Searle and his own critics, I argue that Searle’s ‘internalist’ strategy is unable to deflect Dennett’s combined robotic-systems reply and the allied Total Turing Test (3T). Many would hold that the 3T marks (...) the culmination of the dialectic and, in principle, constitutes a fully adequate empirical standard for judging that an artifact is intelligent on a par with human beings. However, the paper carries the debate forward by arguing that the sociolinguistic factors highlighted in externalist views in the philosophy of language indicate the need for a fundamental shift in perspective in a Truly Total Turing Test (4T). It’s not enough to focus on Dennett’s individual robot viewed as a system; instead, we need to focus on an ongoing system of such artifacts. Hence a 4T should evaluate the general category of cognitive organization under investigation, rather than the performance of single specimens. From this comprehensive standpoint, the question is not whether an individual instance could simulate intelligent behavior within the context of a pre-existing sociolinguistic culture developed by the human cognitive type. Instead the key issue is whether the artificial cognitive type itself is capable of producing a comparable sociolinguistic medium. (shrink)

Mahadevan (2018, AAAI Conference. https://people.cs.umass.edu/~mahadeva/papers/aaai2018-imagination.pdf) proposes that we are at the cusp of imagination science, one of whose primary concerns will be the design of imagination machines. Programs have been written that are capable of generating jokes (Kim Binsted’s JAPE), producing line-drawings that have been exhibited at such galleries as the Tate (Harold Cohen’s AARON), composing music in several styles reminiscent of such greats as Vivaldi and Mozart (David Cope’s Emmy), proving geometry theorems (Herb Gelernter’s IBM program), and inducing quantitative (...) laws from empirical data (Pat Langley, Gary Bradshaw, Jan Zytkow, and Herbert Simon’s BACON). In recent years, Dartmouth has been hosting Turing Tests in creativity in three categories: short stories, sonnets, and dance music DJ sets. In this post, I will provide a brief and non-exhaustive survey of some plausible responses to these imagination machines and the related prospects for our understanding of the imagination. (shrink)

In the last 20 years the Turing test has been left further behind by new developments in artificial intelligence. At the same time, however, these developments have revived some key elements of the Turing test: imitation and adversarialness. On the one hand, many generative models, such as generative adversarial networks, build imitators under an adversarial setting that strongly resembles the Turing test. The term “Turing learning” has been used for this kind of setting. (...) On the other hand, AI benchmarks are suffering an adversarial situation too, with a ‘challenge-solve-and-replace’ evaluation dynamics whenever human performance is ‘imitated’. The particular AI community rushes to replace the old benchmark by a more challenging benchmark, one for which human performance would still be beyond AI. These two phenomena related to the Turing test are sufficiently distinctive, important and general for a detailed analysis. This is the main goal of this paper. After recognising the abyss that appears beyond superhuman performance, we build on Turing learning to identify two different evaluation schemas: Turing testing and adversarial testing. We revisit some of the key questions surrounding the Turing test, such as ‘understanding’, commonsense reasoning and extracting meaning from the world, and explore how the new testing paradigms should work to unmask the limitations of current and future AI. Finally, we discuss how behavioural similarity metrics could be used to create taxonomies for artificial and natural intelligence. Both testing schemas should complete a transition in which humans should give way to machines—not only as references to be imitated but also as judges—when pursuing and measuring machine intelligence. (shrink)

In his 1950 paper “Computing Machinery and Intelligence,” Alan Turing proposed that we can determine whether a machine thinks by considering whether it can win at a simple imitation game. A neutral questioner communicates with two different systems – one a machine and a human being – without knowing which is which. If after some reasonable amount of time the machine is able to fool the questioner into identifying it as the human, the machine wins the game, and we (...) should conclude that it thinks. This imitation game, now known as the Turing Test, has been much discussed by philosophers of mind, and for more than half a century now there has been considerable debate about whether it is an adequate test for thinking. But what has not been much discussed are the sexed presuppositions underlying the test. Too often forgotten in the philosophical discussion is the fact that Turing’s imitation game is modeled on an imitation game in which a neutral questioner communicates with two different humans – one a man and one a woman – without knowing which is which. In this original imitation game, the man wins the game if he is able to fool the questioner into identifying him as the woman. In this paper, I explore the implications of this set-up. As I argue, the fact that the Turing test was modeled on a man/woman imitation game seems to have led us astray in various ways in our attempt to conduct an effective investigation and assessment of computer intelligence. (shrink)