Fast Marginal Likelihood Maximisation for Sparse Bayesian Models

Abstract

The ’sparse Bayesian’ modelling approach, as exemplified by the ’relevance vector machine’, enables sparse classification and regression functions to be obtained by linearlyweighting a small number of fixed basis functions from a large dictionary of potential candidates. Such a model conveys a number of advantages over the related and very popular ’support vector machine’, but the necessary ’training’ procedure - optimisation of the marginal likelihood function is typically much slower. We describe a new and highly accelerated algorithm which exploits recently-elucidated properties of the marginal likelihood function to enable maximisation via a principled and efficient sequential addition and deletion of candidate basis functions.