Mixed-strategy learning with continuous action sets

Perkins, Steven and Mertikopoulos, Panayotis and Leslie, David Stuart (2017) Mixed-strategy learning with continuous action sets. IEEE Transactions on Automatic Control, 62 (1). pp. 379-384. ISSN 0018-9286

PDF (PerkinsMertikopoulosLeslie15)
PerkinsMertikopoulosLeslie_Revision2.pdf - Accepted Version
Available under License Creative Commons Attribution.

Download (283kB)
PDF (07364177)
07364177.pdf - Published Version
Available under License Creative Commons Attribution.

Download (331kB)


Motivated by the recent applications of game-theoretical learning to the design of distributed control systems, we study a class of control problems that can be formulated as potential games with continuous action sets. We propose an actor-critic reinforcement learning algorithm that adapts mixed strategies over continuous action spaces. To analyse the algorithm we extend the theory of finite-dimensional two-timescale stochastic approximation to a Banach space setting, and prove that the continuous dynamics of the process converge to equilibrium in the case of potential games. These results combine to give a provablyconvergent learning algorithm in which players do not need to keep track of the controls selected by other agents.

Item Type:
Journal Article
Journal or Publication Title:
IEEE Transactions on Automatic Control
Additional Information:
©2015 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.
Uncontrolled Keywords:
ID Code:
Deposited By:
Deposited On:
22 Dec 2015 13:04
Last Modified:
22 Nov 2022 02:38