MUMBO : MUlti-task Max-value Bayesian Optimization

Moss, Henry B. and Leslie, David S. and Rayson, Paul (2020) MUMBO : MUlti-task Max-value Bayesian Optimization. In: achine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2020, Proceedings :. Springer, pp. 447-462. ISBN 9783030676636

Text (2006.12093v1)
2006.12093v1.pdf - Accepted Version
Available under License Creative Commons Attribution-NonCommercial.
Download (1MB)

Abstract

We propose MUMBO, the first high-performing yet computationally efficient acquisition function for multi-task Bayesian optimization. Here, the challenge is to perform efficient optimization by evaluating low-cost functions somehow related to our true target function. This is a broad class of problems including the popular task of multi-fidelity optimization. However, while information-theoretic acquisition functions are known to provide state-of-the-art Bayesian optimization, existing implementations for multi-task scenarios have prohibitive computational requirements. Previous acquisition functions have therefore been suitable only for problems with both low-dimensional parameter spaces and function query costs sufficiently large to overshadow very significant optimization overheads. In this work, we derive a novel multi-task version of entropy search, delivering robust performance with low computational overheads across classic optimization challenges and multi-task hyper-parameter tuning. MUMBO is scalable and efficient, allowing multi-task Bayesian optimization to be deployed in problems with rich parameter and fidelity spaces.