Multi-armed bandit for species discovery : A Bayesian nonparametric approach

Battiston, Marco and Favaro, Stefano and Teh, Yee Whye (2018) Multi-armed bandit for species discovery : A Bayesian nonparametric approach. Journal of the American Statistical Association, 113 (521). pp. 455-466. ISSN 0162-1459

Abstract

Let (P1, …, PJ) denote J populations of animals from distinct regions. A priori, it is unknown which species are present in each region and what are their corresponding frequencies. Species are shared among populations and each species can be present in more than one region with its frequency varying across populations. In this article, we consider the problem of sequentially sampling these populations to observe the greatest number of different species. We adopt a Bayesian nonparametric approach and endow (P1, …, PJ) with a hierarchical Pitman–Yor process prior. As a consequence of the hierarchical structure, the J unknown discrete probability measures share the same support, that of their common random base measure. Given this prior choice, we propose a sequential rule that, at every time step, given the information available up to that point, selects the population from which to collect the next observation. Rather than picking the population with the highest posterior estimate of producing a new value, the proposed rule includes a Thompson sampling step to better balance the exploration–exploitation trade-off. We also propose an extension of the algorithm to deal with incidence data, where multiple observations are collected in a time period. The performance of the proposed algorithms is assessed through a simulation study and compared to three other strategies. Finally, we compare these algorithms using a dataset of species of trees, collected from different plots in South America. Supplementary materials for this article are available online.