GIBBON: General-purpose Information-Based Bayesian Optimisation

Moss, Henry B. and Leslie, David S. and Gonzalez, Javier and Rayson, Paul (2021) GIBBON: General-purpose Information-Based Bayesian Optimisation. Journal of Machine Learning Research, 22 (235). pp. 1-49. ISSN 1532-4435

[thumbnail of 21-0120]
Text (21-0120)
21_0120.pdf - Published Version
Available under License Creative Commons Attribution.

Download (2MB)

Abstract

This paper describes a general-purpose extension of max-value entropy search, a popular approach for Bayesian Optimisation (BO). A novel approximation is proposed for the information gain -- an information-theoretic quantity central to solving a range of BO problems, including noisy, multi-fidelity and batch optimisations across both continuous and highly-structured discrete spaces. Previously, these problems have been tackled separately within information-theoretic BO, each requiring a different sophisticated approximation scheme, except for batch BO, for which no computationally-lightweight information-theoretic approach has previously been proposed. GIBBON (General-purpose Information-Based Bayesian OptimisatioN) provides a single principled framework suitable for all the above, out-performing existing approaches whilst incurring substantially lower computational overheads. In addition, GIBBON does not require the problem's search space to be Euclidean and so is the first high-performance yet computationally light-weight acquisition function that supports batch BO over general highly structured input spaces like molecular search and gene design. Moreover, our principled derivation of GIBBON yields a natural interpretation of a popular batch BO heuristic based on determinantal point processes. Finally, we analyse GIBBON across a suite of synthetic benchmark tasks, a molecular search loop, and as part of a challenging batch multi-fidelity framework for problems with controllable experimental noise.

Item Type:
Journal Article
Journal or Publication Title:
Journal of Machine Learning Research
Uncontrolled Keywords:
/dk/atira/pure/subjectarea/asjc/1700/1702
Subjects:
?? artificial intelligencesoftwarestatistics and probabilitycontrol and systems engineering ??
ID Code:
160945
Deposited By:
Deposited On:
13 Oct 2021 14:15
Refereed?:
Yes
Published?:
Published
Last Modified:
03 Jan 2024 00:28