Recurrent policy gradients

Logic Journal of the IGPL 18 (5):620-634 (2010)
  Copy   BIBTEX

Abstract

Reinforcement learning for partially observable Markov decision problems is a challenge as it requires policies with an internal state. Traditional approaches suffer significantly from this shortcoming and usually make strong assumptions on the problem domain such as perfect system models, state-estimators and a Markovian hidden system. Recurrent neural networks offer a natural framework for dealing with policy learning using hidden state and require only few limiting assumptions. As they can be trained well using gradient descent, they are suited for policy gradient approaches. In this paper, we present a policy gradient method, the Recurrent Policy Gradient which constitutes a model-free reinforcement learning method. It is aimed at training limited-memory stochastic policies on problems which require long-term memories of past observations. The approach involves approximating a policy gradient for a recurrent neural network by backpropagating return-weighted characteristic eligibilities through time. Using a ‘‘Long Short-Term Memory’’ RNN architecture, we are able to outperform previous RL methods on three important benchmark tasks. Furthermore, we show that using history-dependent baselines helps reducing estimation variance significantly, thus enabling our approach to tackle more challenging, highly stochastic environments

Other Versions

No versions found

Links

PhilArchive



    Upload a copy of this work     Papers currently archived: 103,388

External links

Setup an account with your affiliations in order to access resources via your University's proxy server

Through your library

Similar books and articles

重点サンプリングを用いた Ga による強化学習.Kimura Hajime Tsuchiya Chikao - 2005 - Transactions of the Japanese Society for Artificial Intelligence 20:1-10.
Deep Learning-Based Speech Emotion Recognition.Sharma Karan - 2022 - International Journal of Multidisciplinary and Scientific Emerging Research 10 (2):715-718.

Analytics

Added to PP
2015-02-04

Downloads
14 (#1,321,670)

6 months
1 (#1,572,794)

Historical graph of downloads
How can I increase my downloads?

Author's Profile

Citations of this work

No citations found.

Add more citations

References found in this work

No references found.

Add more references