Methodology and theory for unbiased Markov chain Monte Carlo and alternatives

Papp, Tamas and Sherlock, Chris (2025) Methodology and theory for unbiased Markov chain Monte Carlo and alternatives. PhD thesis, Lancaster University.

Text (2025PappPhD)
2025PappPhD.pdf - Published Version
Available under License Creative Commons Attribution.
Download (4MB)

Abstract

Markov chain Monte Carlo (MCMC) is the default inference method in Bayesian statistics. However, MCMC algorithms are biased, which can limit the extent to which these algorithms can be parallelized, and which can complicate the subsequent statistical inference. Recently-proposed methods provide ways of entirely eliminating this bias, based on simulating a coupling of two Markov chains that evolve in tandem. This thesis contributes to the unbiased MCMC literature by improving the efficiency and the theoretical understanding of coupling methods. Additionally, it provides a competitive alternative to coupling methods. The first contribution is an effective coupling for the random walk Metropolis algorithm, a widely-used practical MCMC algorithm. We design this coupling with the explicit aim of scalability to high dimensions, and analyze this coupling in a theoretical framework that can quantify the efficiency of couplings in high dimensions. This framework may be useful for designing and analyzing couplings of other MCMC algorithms. The second contribution is a study on tuning the scalar parameters of coupling methods. We argue that the so-called time-lag parameter is crucial to the efficiency and the robustness of these methods. Even though unbiased MCMC estimators can be noisier than standard MCMC ones, we demonstrate how judicious tuning can ensure that unbiased MCMC is nearly as efficient as standard MCMC. The final contribution is a pair of estimators of the squared Euclidean 2-Wasserstein distance, a strong measure of the discrepancy between two distributions. These estimators are based on approximately independent samples from the distributions of interest. We show that the estimators are often upper and lower bounds on the underlying discrepancy, and we demonstrate that they often outperform coupling methods in statistical applications.