Mixed-strategy learning with continuous action sets

Perkins, Steven and Mertikopoulos, Panayotis and Leslie, David Stuart (2017) Mixed-strategy learning with continuous action sets. IEEE Transactions on Automatic Control, 62 (1). pp. 379-384. ISSN 0018-9286

Preview

PDF (PerkinsMertikopoulosLeslie15)
PerkinsMertikopoulosLeslie_Revision2.pdf - Accepted Version
Available under License Creative Commons Attribution.
Download (283kB)

Preview

PDF (07364177)
07364177.pdf - Published Version
Available under License Creative Commons Attribution.
Download (331kB)

Abstract

Motivated by the recent applications of game-theoretical learning to the design of distributed control systems, we study a class of control problems that can be formulated as potential games with continuous action sets. We propose an actor-critic reinforcement learning algorithm that adapts mixed strategies over continuous action spaces. To analyse the algorithm we extend the theory of finite-dimensional two-timescale stochastic approximation to a Banach space setting, and prove that the continuous dynamics of the process converge to equilibrium in the case of potential games. These results combine to give a provablyconvergent learning algorithm in which players do not need to keep track of the controls selected by other agents.