On Thompson Sampling for Smoother-than-Lipschitz Bandits

Grant, James A. and Leslie, David S. (2020) On Thompson Sampling for Smoother-than-Lipschitz Bandits. In: 23rd International Conference on Artificial Intelligence and Statistics :. Proceedings of Machine Learning Research . Proceedings of Machine Learning Research, pp. 2612-2622.

Text (TSSmootherThanLipschitz)
TSSmootherThanLipschitz.pdf - Accepted Version
Available under License Creative Commons Attribution-NonCommercial.
Download (629kB)

Text (2001.02323v1)
2001.02323v1.pdf - Published Version
Download (729kB)

Abstract

Thompson Sampling is a well established approach to bandit and reinforcement learning problems. However its use in continuum armed bandit problems has received relatively little attention. We provide the first bounds on the regret of Thompson Sampling for continuum armed bandits under weak conditions on the function class containing the true function and sub-exponential observation noise. Our bounds are realised by analysis of the eluder dimension, a recently proposed measure of the complexity of a function class, which has been demonstrated to be useful in bounding the Bayesian regret of Thompson Sampling for simpler bandit problems under sub-Gaussian observation noise. We derive a new bound on the eluder dimension for classes of functions with Lipschitz derivatives, and generalise previous analyses in multiple regards.