Bayesian Reinforcement Learning in Markovian and non-Markovian Tasks

Ez-Zizi, Adnane and Farrell, Simon and Leslie, David Stuart (2015) Bayesian Reinforcement Learning in Markovian and non-Markovian Tasks. In: Computational Intelligence, 2015 IEEE Symposium Series on. IEEE, Cape Town, pp. 579-586. ISBN 9781479975600

Full text not available from this repository.


We present a Bayesian reinforcement learning model with a working memory module which can solve some non-Markovian decision processes. The model is tested, and compared against SARSA (lambda), on a standard working-memory task from the psychology literature. Our method uses the Kalman temporal difference framework, And its extension to stochastic state transitions, to give posterior distributions over state-action values. This framework provides a natural mechanism for using reward information to update more than the current state-action pair, and thus negates the use of eligibility traces. Furthermore, the existence of full posterior distributions allows the use of Thompson sampling for action selection, which in turn removes the need to choose an appropriately parameterised action-selection method.

Item Type:
Contribution in Book/Report/Proceedings
ID Code:
Deposited By:
Deposited On:
22 Aug 2016 10:44
Last Modified:
19 Sep 2023 03:25