Lancaster EPrints

Monotone policies and indexability for bi-directional restless bandits

Glazebrook, Kevin and Hodge, D. J. and Kirkbride, Christopher (2013) Monotone policies and indexability for bi-directional restless bandits. Advances in Applied Probability, 45 (1). pp. 51-85. ISSN 0001-8678

Full text not available from this repository.

Abstract

Motivated by a wide range of applications, we consider a development of Whittle's restless bandit model in which project activation requires a state-dependent amount of a key resource, which is assumed to be available at a constant rate. As many projects may be activated at each decision epoch as resource availability allows. We seek a policy for project activation within resource constraints which minimises an aggregate cost rate for the system. Project indices derived from a Lagrangian relaxation of the original problem exist provided the structural requirement of indexability is met. Verification of this property and derivation of the related indices is greatly simplified when the solution of the Lagrangian relaxation has a state monotone structure for each constituent project. We demonstrate that this is indeed the case for a wide range of bidirectional projects in which the project state tends to move in a different direction when it is activated from that in which it moves when passive. This is natural in many application domains in which activation of a project ameliorates its condition, which otherwise tends to deteriorate or deplete. In some cases the state monotonicity required is related to the structure of state transitions, while in others it is also related to the nature of costs. Two numerical studies demonstrate the value of the ideas for the construction of policies for dynamic resource allocation, most especially in contexts which involve a large number of projects.

Item Type: Article
Journal or Publication Title: Advances in Applied Probability
Uncontrolled Keywords: asset management ; Gittins index ; indexability ; inventory management ; Lagrangian relaxation ; machine maintenance ; monotone policy ; stochastic dynamic programming ; restless bandit ; Whittle index
Subjects: H Social Sciences > HB Economic Theory
Departments: Lancaster University Management School > Management Science
ID Code: 55047
Deposited By: ep_importer_pure
Deposited On: 13 Jun 2012 13:40
Refereed?: Yes
Published?: Published
Last Modified: 13 Jun 2014 11:34
Identification Number:
URI: http://eprints.lancs.ac.uk/id/eprint/55047

Actions (login required)

View Item