Lancaster EPrints

On sparse variational methods and the Kullback-Leibler divergence between stochastic processes

Matthews, Alexander G. de G. and Hensman, James and Turner, Richard and Ghahramani, Zoubin (2016) On sparse variational methods and the Kullback-Leibler divergence between stochastic processes. Journal of Machine Learning Research, 51. pp. 231-239. ISSN 1532-4435

[img]
Preview
PDF (sparseKL_AISTATS) - Submitted Version
Available under License ["licenses_description_creative_commons_attribution_noncommercial_4_0_international_license" not defined].

Download (261Kb) | Preview

    Abstract

    The variational framework for learning inducing variables (Titsias, 2009a) has had a large impact on the Gaussian process literature. The framework may be interpreted as minimizing a rigorously defined Kullback-Leibler divergence between the approximating and posterior processes. To our knowledge this connection has thus far gone unremarked in the literature. In this paper we give a substantial generalization of the literature on this topic. We give a new proof of the result for infinite index sets which allows inducing points that are not data points and likelihoods that depend on all function values. We then discuss augmented index sets and show that, contrary to previous works, marginal consistency of augmentation is not enough to guarantee consistency of variational inference with the original model. We then characterize an extra condition where such a guarantee is obtainable. Finally we show how our framework sheds light on interdomain sparse approximations and sparse approximations for Cox processes.

    Item Type: Article
    Journal or Publication Title: Journal of Machine Learning Research
    Subjects:
    Departments: Faculty of Health and Medicine > Medicine
    ID Code: 83543
    Deposited By: ep_importer_pure
    Deposited On: 14 Dec 2016 09:06
    Refereed?: Yes
    Published?: Published
    Last Modified: 09 Dec 2017 00:04
    Identification Number:
    URI: http://eprints.lancs.ac.uk/id/eprint/83543

    Actions (login required)

    View Item