Prospects for bandit solutions in sensor management

Pavlidis, N and Adams, N M and Nicholson, M and Hand, D J (2010) Prospects for bandit solutions in sensor management. The Computer Journal, 53 (9). pp. 1370-1383. ISSN 0010-4620

[thumbnail of Prospects for bandit solutions in sensor management]
PDF (Prospects for bandit solutions in sensor management)
10.pdf - Submitted Version

Download (880kB)


Sensor management in information-rich and dynamic environments can be posed as a sequential action selection problem with side information. To study such problems we employ the dynamic multi-armed bandit with covariates framework. In this generalization of the multi-armed bandit, the expected rewards are time-varying linear functions of the covariate vector. The learning goal is to associate the covariate with the optimal action at each instance, essentially learning to partition the covariate space adaptively. Applications of sensor management are frequently in environments in which the precise nature of the dynamics is unknown. In such settings, the sensor manager tracks the evolving environment by observing only the covariates and the consequences of the selected actions. This creates difficulties not encountered in static problems, and changes the exploitation–exploration dilemma. We study the relationship between the different factors of the problem and provide interesting insights. The impact of the environment dynamics on the action selection problem is influenced by the covariate dimensionality. We present the surprising result that strategies that perform very little or no exploration perform surprisingly well in dynamic environments

Item Type:
Journal Article
Journal or Publication Title:
The Computer Journal
Uncontrolled Keywords:
ID Code:
Deposited By:
Deposited On:
11 Jul 2011 18:36
Last Modified:
16 Sep 2023 00:35