Best-response Dynamics in Zero-sum Stochastic Games

Leslie, David and Perkins, Steven and Xu, Zibo (2020) Best-response Dynamics in Zero-sum Stochastic Games. Journal of Economic Theory, 189: 105095. ISSN 0022-0531

Text (StochasticGamesLearning_JETrevision)
StochasticGamesLearning_JETrevision.pdf - Accepted Version
Available under License Creative Commons Attribution.
Download (382kB)

Abstract

We define and analyse three learning dynamics for two-player zero-sum discounted-payoff stochastic games. A continuous-time best-response dynamic in mixed strategies is proved to converge to the set of Nash equilibrium stationary strategies. Extending this, we introduce a fictitious-play-like process in a continuous-time embedding of a stochastic zero-sum game, which is again shown to converge to the set of Nash equilibrium strategies. Finally, we present a modified δ-converging best-response dynamic, in which the discount rate converges to 1, and the learned value converges to the asymptotic value of the zero-sum stochastic game. The critical feature of all the dynamic processes is a separation of adaption rates: beliefs about the value of states adapt more slowly than the strategies adapt, and in the case of the δ-converging dynamic the discount rate adapts more slowly than everything else.