PyMC began development in 2003, as an effort to generalize the process of building Metropolis-Hastings samplers, with an aim to making Markov chain Monte Carlo (MCMC) more accessible to applied scientists. The choice to develop PyMC as a python module, rather than a standalone application, allowed the use of MCMC methods in a larger modeling framework. By 2005, PyMC was reliable enough for version 1.0 to be released to the public. A small group of regular users, most associated with the University of Georgia, provided much of the feedback necessary for the refinement of PyMC to a usable state.
In 2006, David Huard and Anand Patil joined Chris Fonnesbeck on the development team for PyMC 2.0. This iteration of the software strives for more flexibility, better performance and a better end-user experience than any previous version of PyMC. PyMC 2.2 was released in April 2012. It contained numerous bugfixes and optimizations, as well as a few new features, including improved output plotting, csv table output, improved imputation syntax, and posterior predictive check plots. PyMC 2.3 was released on October 31, 2013. It included Python 3 compatibility, improved summary plots, and some important bug fixes.
In 2011, John Salvatier began thinking about implementing gradient-based MCMC samplers, and developed the
mcex package to experiment with his ideas. The following year, John was invited by the team to re-engineer PyMC to accomodate Hamiltonian Monte Carlo sampling. This led to the adoption of Theano as the computational back end, and marked the beginning of PyMC’s development. The first alpha version of PyMC was released in June 2015. Over the following 2 years, the core development team grew to 12 members, and the first release, PyMC 3.0, was launched in January 2017. In 2020 the PyMC developers forked Theano and in 2021 renamed the forked project to Aesara.
The third major version of PyMC benefitted from being re-written from scratch. Substantial improvements in the user interface and performance resulted from this. While PyMC2 relied on Fortran extensions (via f2py) for most of the computational heavy-lifting, PyMC leverages Aesara, a fork of the Theano library from the Montréal Institute for Learning Algorithms (MILA), for array-based expression evaluation, to perform its computation. What this provided, above all else, is fast automatic differentiation, which is at the heart of the gradient-based sampling and optimization methods providing inference for probabilistic programming.
Major changes from previous versions:
New flexible object model and syntax (not backward-compatible with PyMC2).
Gradient-based MCMC methods, including Hamiltonian Monte Carlo (HMC), the No U-turn Sampler (NUTS), and Stein Variational Gradient Descent.
Variational inference methods, including automatic differentiation variational inference (ADVI) and operator variational inference (OPVI).
An interface for easy formula-based specification of generalized linear models (GLM).
Elliptical slice sampling.
Specialized distributions for representing time series.
A library of Jupyter notebooks that provide case studies and fully developed usage examples.
While the addition of Aesara added a level of complexity to the development of PyMC, fundamentally altering how the underlying computation is performed, the dev team worked hard to maintain the elegant simplicity of the original PyMC model specification syntax.
TODO: explanation on v4 development from history side, then link to PyMC 4.0 Release Announcement for the explanation about the actual new features and improvements