Animals routinely adapt to changes in the environment in order to survive. Though reinforcement learning may play a role in such adaptation, it is not clear that it is the only mechanism involved, as it is not well suited to producing rapid, relatively immediate changes in strategies in response to environmental changes. This research proposes that counterfactual reasoning might be an additional mechanism that facilitates change detection. An experiment is conducted in which a task state changes over time and the (...) participants had to detect the changes in order to perform well and gain monetary rewards. A cognitive model is constructed that incorporates reinforcement learning with counterfactual reasoning to help quickly adjust the utility of task strategies in response to changes. The results show that the model can accurately explain human data and that counterfactual reasoning is key to reproducing the various effects observed in this change detection paradigm. (shrink)

There is an apparent connection between reinforcement learning and agency. Artificial entities controlled by reinforcement learning algorithms are standardly referred to as agents, and the mainstream view in the psychology and neuroscience of agency is that humans and other animals are reinforcement learners. This article examines this connection, focusing on artificial reinforcement learning systems and assuming that there are various forms of agency. Artificial reinforcement learning systems satisfy plausible conditions for minimal agency, and those which use models of the environment (...) to perform forward search are capable of a form of agency which may reasonably be called action for reasons. (shrink)

We investigate how Reinforcement Learning methods can be used to solve the production selection and production ordering problem in ACT‐R. We focus on four algorithms from the Q learning family, tabular Q and three versions of Deep Q Networks, as well as the ACT‐R utility learning algorithm, which provides a baseline for the Q algorithms. We compare the performance of these five algorithms in a range of lexical decision tasks framed as sequential decision problems.

We consider the efficacy of various forms of reinforcement learning with punishment in evolving linguistic conventions in the context of Lewis-Skyrms signalling games. We show that the learning strategy of reinforcement with iterative punishment is highly effective at evolving optimal conventions in even complex signalling games. It is also robust and can be easily extended to a self-tuning variety of reinforcement learning. We briefly discuss some of the virtues of reinforcement with iterative punishment and how it may be related to (...) learning in nature. (shrink)

Many philosophers are building a solid case in favour of the knowledge account of assertion (KAA). According to KAA, if one asserts that P one represents oneself as knowing that P. KAA has recently received support from linguistic data about prompting challenges, parenthetical positioning and predictions. In this article, I add another argument to this rapidly growing list: an argument from what I will call ‘reinforcing parenthesis’.

Some philosophers have argued that evidence of underlying mechanisms does not provide evidence for the effectiveness of a medical intervention. One such argument appeals to the unreliability of mechanistic reasoning. However, mechanistic reasoning is not the only way that evidence of mechanisms might provide evidence of effectiveness. A more reliable type of reasoning may be distinguished by appealing to recent work on evidential pluralism in the epistemology of medicine. A case study from virology provides an example of this so‐called reinforced (...) reasoning in medicine. It is argued that in this case study, the available evidence of underlying mechanisms did in fact play a role in providing evidence in favour of a medical intervention. This paper therefore adds a novel and recent case study to the literature in support of evidential pluralism in medicine. (shrink)

In this opinionated review, I draw attention to some of the contributions reinforcement learning can make to questions in the philosophy of mind. In particular, I highlight reinforcement learning's foundational emphasis on the role of reward in agent learning, and canvass two ways in which the framework may advance our understanding of perception and motivation.

Shared intention normally leads to joint action. It does this, it is commonly said, only because it is a characteristically stable phenomenon, a phenomenon that tends to persist from the time it is formed until the time it is fulfilled. However, the issue of what the stability of shared intention comes down to remains largely undertheorized. My aim in this paper is to remedy this shortcoming. I argue that shared intention is a source of moral and epistemic reasons, that responsiveness (...) to such reasons on the part of each individual reinforces her own relevant attitudes, and that this enhances the stability of the shared intention as a whole. We thus discover an important self-reinforcing and stability-enhancing mechanism at the heart of shared intention and action. It follows from this proposal that the psychological commitment that an individual exhibits in joint action is normally a function not only of her own participatory intention, as often maintained, but also of her endorsement of the reasons created by her group’s shared intention to so act. (shrink)

Rapid and precise air operation mission planning is a key technology in unmanned aerial vehicles autonomous combat in battles. In this paper, an end-to-end UAV intelligent mission planning method based on deep reinforcement learning is proposed to solve the shortcomings of the traditional intelligent optimization algorithm, such as relying on simple, static, low-dimensional scenarios, and poor scalability. Specifically, the suppression of enemy air defense mission planning is described as a sequential decision-making problem and formalized as a Markov decision process. Then, (...) the SEAD intelligent planning model based on the proximal policy optimization algorithm is established and a general intelligent planning architecture is proposed. Furthermore, three policy training tricks, i.e., domain randomization, maximizing policy entropy, and underlying network parameter sharing, are introduced to improve the learning performance and generalizability of PPO. Experiments results show that the model in this work is efficient and stable, and can be adapted to the unknown continuous high-dimensional environment. It can be concluded that the UAV intelligent mission planning model based on DRL has powerful intelligent planning performance, and provides a new idea for researching UAV autonomy. (shrink)

Rachlin rightly highlights behavioural reinforcement, conditional cooperation, and framing. However, genes may explain part of the variance in altruistic behaviour. Framing cannot be used to support his theory of altruism. Reinforcement of acts is not identical to reinforcement of patterns of acts. Further, many patterns of acts could be reinforced, and Rachlin's altruism is not the most likely candidate.

Behaving ethically depends on the ability to recognize that ethical issues exist, to see from an ethical point of view. This ability to see and respond ethically may be related more to attributes of corporate culture than to attributes of individual employees. Efforts to increase ethical standards and decrease pressure to behave unethically should therefore concentrate on the organization and its culture. The purpose of this paper is to discuss how total quality (TQ) techniques can facilitate the development of a (...) cooperative corporate culture that promotes and encourages ethical behavior throughout an organization. (shrink)

Like theories, reconstructions of episodes in the history of science can possess, or lack, certain virtues such that, when we face two or more different reconstructions of the same episode, we assume that we should choose the most “virtuous one”. However, we will argue that, with dissimilar reconstructions of the same episode, it is not always necessary to separate the “good ones” from the “wrong ones”, and that, as a matter of fact, each reconstruction could provide different but perhaps equally (...) relevant data about the episode, about science in general, and about particular philosophical theses. In order to help us to identify these benefits, we will present a criterion that guides the search for historiographical reinforcement of philosophical theses and we will use it to evaluate three different reconstructions of the same scientific episode. (shrink)

The area of smart power grids needs to constantly improve its efficiency and resilience, to provide high quality electrical power in a resilient grid, while managing faults and avoiding failures. Achieving this requires high component reliability, adequate maintenance, and a studied failure occurrence. Correct system operation involves those activities and novel methodologies to detect, classify, and isolate faults and failures and model and simulate processes with predictive algorithms and analytics. In this paper, we showcase the application of a complex-adaptive, self-organizing (...) modeling method, and Probabilistic Boolean Networks, as a way towards the understanding of the dynamics of smart grid devices, and to model and characterize their behavior. This work demonstrates that PBNs are equivalent to the standard Reinforcement Learning Cycle, in which the agent/model has an interaction with its environment and receives feedback from it in the form of a reward signal. Different reward structures were created to characterize preferred behavior. This information can be used to guide the PBN to avoid fault conditions and failures. (shrink)

Sequential action makes up the bulk of human daily activity, and yet much remains unknown about how people learn such actions. In one motor learning paradigm, the serial reaction time (SRT) task, people are taught a consistent sequence of button presses by cueing them with the next target response. However, the SRT task only records keypress response times to a cued target, and thus it cannot reveal the full time‐course of motion, including predictive movements. This paper describes a mouse movement (...) trajectory SRT task in which the cursor must be moved to a cued location. We replicated keypress SRT results, but also found that predictive movement—before the next cue appears—increased during the experiment. Moreover, trajectory analyses revealed that people developed a centering strategy under uncertainty. In a second experiment, we made prediction explicit, no longer cueing targets. Thus, participants had to explore the response alternatives and learn via reinforcement, receiving rewards and penalties for correct and incorrect actions, respectively. Participants were not told whether the sequence of stimuli was deterministic, nor if it would repeat, nor how long it was. Given the difficulty of the task, it is unsurprising that some learners performed poorly. However, many learners performed remarkably well, and some acquired the full 10‐item sequence within 10 repetitions. Comparing the high‐ and low‐performers’ detailed results in this reinforcement learning (RL) task with the first experiment's cued trajectory SRT task, we found similarities between the two tasks, suggesting that the effects in Experiment 1 are due to predictive, rather than reactive processes. Finally, we found that two standard model‐free reinforcement learning models fit the high‐performing participants, while the four low‐performing participants provide better fit with a simple negative recency bias model. (shrink)

Sequential action makes up the bulk of human daily activity, and yet much remains unknown about how people learn such actions. In one motor learning paradigm, the serial reaction time (SRT) task, people are taught a consistent sequence of button presses by cueing them with the next target response. However, the SRT task only records keypress response times to a cued target, and thus it cannot reveal the full time‐course of motion, including predictive movements. This paper describes a mouse movement (...) trajectory SRT task in which the cursor must be moved to a cued location. We replicated keypress SRT results, but also found that predictive movement—before the next cue appears—increased during the experiment. Moreover, trajectory analyses revealed that people developed a centering strategy under uncertainty. In a second experiment, we made prediction explicit, no longer cueing targets. Thus, participants had to explore the response alternatives and learn via reinforcement, receiving rewards and penalties for correct and incorrect actions, respectively. Participants were not told whether the sequence of stimuli was deterministic, nor if it would repeat, nor how long it was. Given the difficulty of the task, it is unsurprising that some learners performed poorly. However, many learners performed remarkably well, and some acquired the full 10‐item sequence within 10 repetitions. Comparing the high‐ and low‐performers’ detailed results in this reinforcement learning (RL) task with the first experiment's cued trajectory SRT task, we found similarities between the two tasks, suggesting that the effects in Experiment 1 are due to predictive, rather than reactive processes. Finally, we found that two standard model‐free reinforcement learning models fit the high‐performing participants, while the four low‐performing participants provide better fit with a simple negative recency bias model. (shrink)

Autonomous underwater vehicles are widely used to accomplish various missions in the complex marine environment; the design of a control system for AUVs is particularly difficult due to the high nonlinearity, variations in hydrodynamic coefficients, and external force from ocean currents. In this paper, we propose a controller based on deep reinforcement learning in a simulation environment for studying the control performance of the vectored thruster AUV. RL is an important method of artificial intelligence that can learn behavior through trial-and-error (...) interactions with the environment, so it does not need to provide an accurate AUV control model that is very hard to establish. The proposed RL algorithm only uses the information that can be measured by sensors inside the AUVs as the input parameters, and the outputs of the designed controller are the continuous control actions, which are the commands that are set to the vectored thruster. Moreover, a reward function is developed for deep RL controller considering different factors which actually affect the control accuracy of AUV navigation control. To confirm the algorithm’s effectiveness, a series of simulations are carried out in the designed simulation environment, which is a method to save time and improve efficiency. Simulation results prove the feasibility of the deep RL algorithm applied to the control system for AUV. Furthermore, our work also provides an optional method for robot control problems to deal with improving technology requirements and complicated application environments. (shrink)

The concept of ‘human dignity’ sits at the heart of international human rights law and a growing number of national constitutions and yet its meaning is heavily contested and contingent. I aim to supplement the theoretical literature on dignity by providing an empirical study of how the concept is used in the specific context of legal discourse on sex work. I will analyse jurisprudence in which commercial sex was declared as incompatible with human dignity, focussing on the South African Constitutional (...) Court case of S v Jordan and the Indian Supreme Court case of Budhadev Karmaskar v State of West Bengal. I will consider how these courts conceptualise dignity and argue that their conclusions on the undignified nature of sex work are predicated on particular sexual norms that privilege emotional and relational intimacy. In light of the stigma faced by sex workers I will explore how a discourse, proclaiming sex work as beneath human dignity, may impact on the way that sex workers are perceived and represented culturally, arguing that it reinforces stigma. I will go on to examine how sex workers subvert the notion that commercial sex is undignified, and resist stigma, by campaigning for the right to sell sex with dignity. I will demonstrate that an alternative legal approach to dignity and sex work is possible, where the two are not considered as inherently incompatible, concluding with thoughts on the risks and benefits of using ‘dignity talk’ in activism and campaigns for sex work law reform. (shrink)

Rolls presents a theory of emotion based on the premise that emotions are evoked by events that are capable of being instrumental reinforcers and punishers. As support for this theory is drawn almost entirely from experiments in non-human primates, valuable insights into the relationship between punishment and reinforcement systems, and the nature of instrumentality, may have been overlooked.

A deep reinforcement learning-based computational guidance method is presented, which is used to identify and resolve the problem of collision avoidance for a variable number of fixed-wing UAVs in limited airspace. The cooperative guidance process is first analyzed for multiple aircraft by formulating flight scenarios using multiagent Markov game theory and solving it by machine learning algorithm. Furthermore, a self-learning framework is established by using the actor-critic model, which is proposed to train collision avoidance decision-making neural networks. To achieve higher (...) scalability, the neural network is customized to incorporate long short-term memory networks, and a coordination strategy is given. Additionally, a simulator suitable for multiagent high-density route scene is designed for validation, in which all UAVs run the proposed algorithm onboard. Simulated experiment results from several case studies show that the real-time guidance algorithm can reduce the collision probability of multiple UAVs in flight effectively even with a large number of aircraft. (shrink)

Interoception, the ability to feel the body’s internal sensations, is an essential aspect of emotional experience. There is mounting evidence that interoception is impaired in common mental health disorders and that poor interoceptive awareness is a major contributor to emotional reactivity, calling for clinical interventions to address this deficit. The manuscript presents a comprehensive theoretical review, drawing on multidisciplinary findings to propose a metatheory of reinforcement mechanisms applicable across a wide range of disorders. We present a reconsideration of operant conditioning (...) through the co-emergence model of reinforcement, which is a neurophenomenological account of the interaction between cognition and interoception, and its consequences on behavior. The model suggests that during memory processing, the retrieval of autobiographical memory (including maladaptive cognition) is dependent upon its co-emerging interoceptive cues occurring at the encoding, consolidation and reconsolidation stages. Accordingly, “interoceptive reinforcement” during emotional distress is a common factor to all emotional disorders and a major cause for relapse. We propose that interoceptive desensitization has transdiagnostic benefits, readily achievable through the cultivation of equanimity during mindfulness training and can be integrated in cognitive and behavioral interventions to permit a transdiagnostic applicability. We summarize the contributions of this approach into 10 specific and testable propositions. (shrink)

A fundamental question in reading research concerns whether attention is allocated strictly serially, supporting lexical processing of one word at a time, or in parallel, supporting concurrent lexical processing of two or more words (Reichle, Liversedge, Pollatsek, & Rayner, 2009). The origins of this debate are reviewed. We then report three simulations to address this question using artificial reading agents (Liu & Reichle, 2010; Reichle & Laurent, 2006) that learn to dynamically allocate attention to 1–4 words to “read” as efficiently (...) as possible. These simulation results indicate that the agents strongly preferred serial word processing, although they occasionally attended to more than one word concurrently. The reason for this preference is discussed, along with implications for the debate about how humans allocate attention during reading. (shrink)