# Posts by Maxim Kochurov

## Gaussian Processes: HSGP Advanced Usage

- 28 June 2024

The Hilbert Space Gaussian processes approximation is a low-rank GP approximation that is particularly well-suited to usage in probabilistic programming languages like PyMC. It approximates the GP using a pre-computed and fixed set of basis functions that don’t depend on the form of the covariance kernel or its hyperparameters. It’s a *parametric* approximation, so prediction in PyMC can be done as one would with a linear model via `pm.Data`

or `pm.set_data`

. You don’t need to define the `.conditional`

distribution that non-parameteric GPs rely on. This makes it *much* easier to integrate an HSGP, instead of a GP, into your existing PyMC model. Additionally, unlike many other GP approximations, HSGPs can be used anywhere within a model and with any likelihood function.

## Introduction to Variational Inference with PyMC

- 13 January 2023

The most common strategy for computing posterior quantities of Bayesian models is via sampling, particularly Markov chain Monte Carlo (MCMC) algorithms. While sampling algorithms and associated computing have continually improved in performance and efficiency, MCMC methods still scale poorly with data size, and become prohibitive for more than a few thousand observations. A more scalable alternative to sampling is variational inference (VI), which re-frames the problem of computing the posterior distribution as an optimization problem.

## Empirical Approximation overview

- 13 January 2023

For most models we use sampling MCMC algorithms like Metropolis or NUTS. In PyMC we got used to store traces of MCMC samples and then do analysis using them. There is a similar concept for the variational inference submodule in PyMC: *Empirical*. This type of approximation stores particles for the SVGD sampler. There is no difference between independent SVGD particles and MCMC samples. *Empirical* acts as a bridge between MCMC sampling output and full-fledged VI utils like `apply_replacements`

or `sample_node`

. For the interface description, see variational_api_quickstart. Here we will just focus on `Emprical`

and give an overview of specific things for the *Empirical* approximation.