Selecting Multiple Web Adverts - a Contextual Multi-armed Bandit with State Uncertainty

Leslie, David Stuart and Edwards, James Anthony (2020) Selecting Multiple Web Adverts - a Contextual Multi-armed Bandit with State Uncertainty. Journal of the Operational Research Society, 71 (1). pp. 100-116. ISSN 0160-5682

[thumbnail of EdwardsLeslieJORS_accepted]
PDF (EdwardsLeslieJORS_accepted)
jors_accepted.pdf - Accepted Version
Available under License Creative Commons Attribution-NonCommercial.

Download (808kB)


We present a method to solve the problem of choosing a set of adverts to display to each of a sequence of web users. The objective is to maximise user clicks over time and to do so we must learn about the quality of each advert in an online manner by observing user clicks. We formulate the problem as a novel variant of a contextual combinatorial multi-armed bandit problem. The context takes the form of a probability distribution over the user's latent topic preference, and rewards are a particular nonlinear function of the selected set and the context. These features ensure that optimal sets of adverts are appropriately diverse. We give a flexible solution method which combines submodular optimisation with existing bandit index policies. User state uncertainty creates ambiguity in interpreting user feedback which prohibits exact Bayesian updating, but we give an approximate method that is shown to work well.

Item Type:
Journal Article
Journal or Publication Title:
Journal of the Operational Research Society
Additional Information:
This is an Accepted Manuscript of an article published by Taylor & Francis in Journal of the Operational Research Society on 20 Feb 2019 available online:
Uncontrolled Keywords:
?? management information systemsstrategy and managementmanagement science and operations researchmarketing ??
ID Code:
Deposited By:
Deposited On:
18 Oct 2018 10:36
Last Modified:
15 Jul 2024 18:27