Online controlled experiments, also known as A/B tests, have become ubiquitous. While many practical challenges in running experiments at scale have been thoroughly discussed, the ethical dimension of A/B testing has been neglected. This article fills this gap in the literature by introducing a new, soft ethics and governance framework that explicitly recognizes how the rise of an experimentation culture in industry settings brings not only unprecedented opportunities to businesses but also significant responsibilities. More precisely, the article (a) introduces a (...) set of principles to encourage ethical and responsible experimentation to protect users, customers, and society; (b) argues that ensuring compliance with the proposed principles is a complex challenge unlikely to be addressed by resorting to a one-solution response; (c) discusses the relevance and effectiveness of several mechanisms and policies in educating, governing, and incentivizing companies conducting online controlled experiments; and (d) offers a list of prompting questions specifically designed to help and empower practitioners by stimulating specific ethical deliberations and facilitating coordination among different groups of stakeholders. (shrink)

This paper examines a constellation of ethical and editorial issues that have arisen since philosophers started to conduct, submit and publish empirical research. These issues encompass concerns over responsible authorship, fair treatment of human subjects, ethicality of experimental procedures, availability of data, unselective reporting and publishability of research findings. This study aims to assess whether the philosophical community has as yet successfully addressed such issues. To do so, the instructions for authors, submission process and published research papers of 29 main (...) journals in philosophy have been considered and analyzed. In light of the evidence reported here, it is argued that the philosophical community has as yet failed to properly tackle such issues. The paper also delivers some recommendations for authors, reviewers and editors in the field. (shrink)

In this paper, I provide a qualified defense of the claim that cognitive biases are not necessarily signs of irrationality, but rather the result of using normative standards that are too narrow. I show that under certain circumstances, behavior that violates traditional norms of rationality can be adaptive. Yet, I express some reservations about the claim that we should replace our traditional normative standards. Furthermore, I throw doubt on the claim that the replacement of normative standards would license optimistic verdicts (...) about human rationality. (shrink)

Recent replication crises in psychology and other fields have led to intense reflection about the validity of common research practices. Much of this reflection has focussed on reporting standards, and how they may be related to the questionable research practices that could underlie a high proportion of irreproducible findings in the published record. As a developing field, it is particularly important for Experimental Philosophy to avoid some of the pitfalls that have beset other disciplines. To this end, here we provide (...) a detailed, comprehensive assessment of current reporting practices in Experimental Philosophy. We focus on the quality of statistical reporting and the disclosure of information about study methodology. We assess all the articles using quantitative methods that were published over the years 2013–2016 in 29 leading philosophy journals. We find that null hypothesis significance testing is the prevalent statistical practice in Experimental Philosophy, although relying solely on this approach has been criticised in the psychological literature. To augment this approach, various additional measures have become commonplace in other fields, but we find that Experimental Philosophy has adopted these only partially: 53% of the papers report an effect size, 28% confidence intervals, 1% examined prospective statistical power and 5% report observed statistical power. Importantly, we find no direct relation between an article’s reporting quality and its impact. We conclude with recommendations for authors, reviewers and editors in Experimental Philosophy, to facilitate making research statistically-transparent and reproducible. (shrink)

De Marchi and Lorenzetti :253-261, 2016) have recently argued that in fields where the journal impact factor is not calculated, such as in the humanities, it is key to find other indicators that would allow the relevant community to assess the quality of scholarly journals and the research outputs that are published in them. The authors' suggestion is that information concerning the journal's rejection rate and the number of subscriptions sold is important and should be used for such assessment. The (...) question addressed by the authors is very important, yet their proposed solutions are problematic. Here I point to some of these problems and illustrate them by considering as a case in point the field of philosophy. Specifically, here I argue for four main claims. First, even assuming that IF provides a reliable indicator of the quality of journals for the assessment of research outputs, De Marchi and Lorenzetti have failed to validate their suggested indicators and proxies. Second, it has not been clarified why, in absence of IF, other journal-based metrics that are currently available should not be used. Third, the relationship between IF and rejection rate is more complex than the authors suggest. Fourth, accepting the number of sold subscriptions as a proxy would result in discrimination against open access journals. The upshot of my analysis is that the question of how to assess journals and research outputs in the humanities is still far from resolved. (shrink)

This paper discusses Stanovich's appeal to individual differences in reasoning and decision-making to undermine the “adaptive rationality” project put forth by Gigerenzer and his co-workers. I discuss two different arguments based on Stanovich's research. First, heterogeneity in the use of heuristics seems to be at odds with the adaptationist background of the project. Second, the existence of correlations between cognitive ability and susceptibility to cognitive bias suggests that the “standard picture of rationality” (Stein, 1996, 4) is normatively adequate. I argue (...) that, as matters stand, none of the arguments can be seen as fully compelling. Nevertheless, my discussion is not only critical of Stanovich's research, as I also show that (and how) his research can push forward the so-called “rationality debate” by encouraging greater theoretical and experimental work. (shrink)

Adaptive rationality theorists question the manner in which psychologists have typically assessed rational behavior and cognition. According to them, human rationality is adaptive, and the biases reported in the psychological literature are best seen as the result of using normative standards that are too narrow. As it turns out, their challenge is also quite controversial, and several aspects of it have been called into question. Yet, whilst it is often suggested that the lack of cogency comes about due to the (...) implausibility of the alternative normative framework, in this paper I articulate a different strategy to resist the revolutionary rhetoric of AR. As I argue here, even if we accept the normative framework of AR, the challenge from AR is less damaging than usually accepted. In particular, I challenge the claim that biases reported in the literature should be conceived of as violations of axiomatic rationality. I argue that the category of bias refers instead to a range of heterogeneous phenomena and that, since several important families of biases are not just violations of axiomatic rationality, these are not vulnerable to the AR challenge. In fact, I also show that the families I consider here look like plausible cases of irrational behavior from the perspective of AR, and that the outcome of my analysis does not sit well with AR theorists’ claim that people are generally successful at achieving prudential and epistemic goals. (shrink)

Gigerenzer’s ‘external validity argument’ plays a pivotal role in his critique of the heuristics and biases research program (HB). The basic idea is that (a) the experimental contexts deployed by HB are not representative of the real environment and that (b) the differences between the setting and the real environment are causally relevant, because they result in different performances by the subjects. However, by considering Gigerenzer’s work on frequencies in probability judgments, this essay attempts to show that there are fatal (...) flaws in the argument. Specifically, each of the claims is controversial: whereas (b) is not adequately empirically justified, (a) is inconsistent with the ‘debiasing’ program of Gigerenzer’s ABC group. Therefore, whatever reason we might have for believing that the experimental findings of HB lack experimental validity, this should not be based on Gigerenzer’s version of the argument. (shrink)

Naturalistic philosophers rely on literature search and review in a number of ways and for different purposes. Yet this article shows how processes of literature search and review are likely to be affected by widespread and systematic biases. A solution to this problem is offered here. Whilst the tradition of systematic reviews of literature from scientific disciplines has been neglected in philosophy, systematic reviews are important tools that minimize bias in literature search and review and allow for greater reproducibility and (...) transparency. If naturalistic philosophers wish to reduce bias in their research, they should then supplement their traditional tools for literature search and review by including systematic methodologies. (shrink)

One of the most fascinating topics of current investigation in the literature on judgment and decision-making concerns the exploration of foreign language effects (henceforth, FLE). Specifically, recent research suggests that presenting information in a foreign language helps reasoners make better choices. However, this piece aims at making scholars aware of a blind spot in this stream of research. In particular, research on FLE has imported only one view of judgment and decision-making, in which the heuristics that people use are seen (...) as conducive to biases and, in turn, to costly mistakes. But heuristics are not necessarily a liability, and this article indicates two routes to push forward research on FLE in judgment and decision-making. First, research on FLE should be expanded to explore also classes of fast and frugal heuristics, which have been shown to lead to accurate predictions in several contexts characterized by uncertainty. Second, research on FLE should be open to challenge the interpretations given to previous FLE findings, since alternative accounts are plausible and not ruled out by evidence. (shrink)

Naturalistic philosophers rely on literature search and review in a number of ways and for different purposes. Yet this article shows how processes of literature search and review are likely to be affected by widespread and systematic biases. A solution to this problem is offered here. Whilst the tradition of systematic reviews of literature from scientific disciplines has been neglected in philosophy, systematic reviews are important tools that minimize bias in literature search and review and allow for greater reproducibility and (...) transparency. If naturalistic philosophers wish to reduce bias in their research, they should then supplement their traditional tools for literature search and review by including systematic methodologies. (shrink)

How much irrationality should we ascribe to human cognition? This is the question at the heart of the “rationality debate,” which in the main originated as a reaction to the large body of research...

How much irrationality should we ascribe to human cognition? Psychological evidence suggests that people’s reasoning is largely inaccurate, but according to an evolutionary argument for rationality, we have good reasons to believe that this is not so. To solve the conflict between psychological evidence and EAR, commentators have usually put the blame either on the psychological evidence, arguing that inaccurate reasoning appears only in the context of lab studies, or on the premises of EAR, charged with not being in line (...) with the concepts and findings of evolutionary biology. I argue that Hammond’s distinction between two distinct criteria of rationality, namely coherence and correspondence, might shed new light on this apparent conflict. I show that EAR might be interpreted in two different ways, and that EAR and psychological evidence might in fact be both correct if they appeal to different criteria of accurate reasoning. Moreover, evolutionary considerations have been recently used not to oppose the existence of violations of norms of coherence but rather to explain it. (shrink)

In a series of recent publications, Gigerenzer and his collaborators have attempted to derive new norms of rationality from their psychological research in the Centre for Adaptive Behaviour and Cognition (ABC). Specifically, they have claimed that there are good reasons to replace the norms traditionally used to assess rational behaviour, which rest on the ideal of optimisation. Their proposal has considerable importance, as it has been laid out as a revision of the normative framework accepted in the social, behavioural, and (...) cognitive sciences. Still, whereas the ABC scholars present their approach as diametrically opposed to the framework of optimisation—two incompatible takes on the problem of defining rational behaviour—this paper argues that it is not entirely clear whether this is the case. I introduce a distinction between different kinds of reflection upon norms of rationality that has been neglected by ABC scholars, and provide reasons to think that the departure from the traditional framework might be less radical than ABC scholars suppose. (shrink)

Mark Kelman’s recent book, The Heuristics Debate (HD), has two main goals. First, it seeks to reconstruct the controversy in decision science between Kahneman et al.’s heuristics-and-biases approach and Gigerenzer et al.’s fast-and-frugal heuristics approach. Second, it tries to discuss its implications for jurisprudence and policy-making. This study focuses on the first task only. The study attempts to show that, although HD has several important merits, its interpretation of the controversy misses some crucial aspects. Specifically, HD fails to appreciate that (...) the debate is fundamentally about what a “rational” judgment is in the first place. Moreover, because of this, HD also fails to acknowledge the interplay between normative and methodological considerations. With regard to this aspect, HD’s treatment of the controversy fits into a long tradition. This study takes the opportunity to rectify the error. (shrink)

This article is concerned with major current developments in moral psychology, deriving from the study of the neurobiological bases of our responses to moral dilemmas. I briefly illustrate the most important research programs and outline the burning issues in neuroethics, both empirical and conceptual.

The idea that humans are prone to widespread and systematic biases has dominated the psychological study of thinking and decision-making. The conclusion that has often been drawn is that people are irrational. In recent decades, however, a number of psychologists have started to call into question key claims and findings in research on human biases. In particular, a body of research has come together under the heading of adaptive rationality. AR theorists argue that people should not be assessed against formal (...) principles of rationality but rather against the goals they entertain. Moreover, AR theorists maintain that the conclusion that people are irrational is unsupported: people are often remarkably successful once assessed against their goals and given the cognitive and external constraints imposed by the environment. The growth of literature around AR is what motivates the present investigation, and assessing the plausibility of the AR challenge to research on human biases is the goal of this thesis. My enquiry analyses several aspects of this suggested turn in the empirical study of rationality and provides one of the first philosophically-informed appraisals of the prospects of AR. First and foremost, my thesis seeks to provide a qualified defence of the AR project. On the one hand, I agree with AR theorists that there is room for a conceptual revolution in the study of thinking and decision-making: while it is commonly argued that behaviour and cognition should be assessed against formal principles of rationality, I stress the importance of assessing behaviour against the goals that people entertain. However, I also contend that AR theorists have hitherto failed to provide compelling evidence in support of their most ambitious and optimistic theses about people’s rationality. In particular, I present a great deal of evidence suggesting that people are often unsuccessful at achieving prudential and epistemic goals and I argue that AR theorists have not made clear how, in light of this evidence, optimistic claims about human rationality could be defended. (shrink)