QuickMMCTest : quick multiple Monte Carlo testing

Gandy, Axel and Hahn, Georg (2017) QuickMMCTest : quick multiple Monte Carlo testing. Statistics and Computing, 27 (3). pp. 823-832. ISSN 0960-3174

[thumbnail of quick]
PDF (quick)
quick.pdf - Accepted Version
Available under License Creative Commons Attribution-NonCommercial.

Download (144kB)


Multiple hypothesis testing is widely used to evaluate scientific studies involving statistical tests. However, for many of these tests, p values are not available and are thus often approximated using Monte Carlo tests such as permutation tests or bootstrap tests. This article presents a simple algorithm based on Thompson Sampling to test multiple hypotheses. It works with arbitrary multiple testing procedures, in particular with step-up and step-down procedures. Its main feature is to sequentially allocate Monte Carlo effort, generating more Monte Carlo samples for tests whose decisions are so far less certain. A simulation study demonstrates that for a low computational effort, the new approach yields a higher power and a higher degree of reproducibility of its results than previously suggested methods.

Item Type:
Journal Article
Journal or Publication Title:
Statistics and Computing
Additional Information:
© Springer Science+Business Media New York 2016. The final publication is available at Springer via http://dx.doi.org/10.1007/s11222-016-9656-z
Uncontrolled Keywords:
?? multiple hypothesis testingmonte carlothompson samplingbonferroni correctionbenjamini-hochberg procedurefalse discovery ratebonferroni procedureassociationvaluescomputational theory and mathematicstheoretical computer sciencestatistics and probabilitystat ??
ID Code:
Deposited By:
Deposited On:
15 Jan 2018 11:56
Last Modified:
18 Dec 2023 01:50