Implementations¶

Latent Gaussian process. 

Marginal Gaussian process. 

Latent Gaussian process whose covariance is a tensor product kernel. 

Marginal Gaussian process whose covariance is a tensor product kernel. 

Approximate marginal Gaussian process. 

Student's T process prior. 
 class pymc3.gp.gp.Latent(mean_func=<pymc3.gp.mean.Zero object>, cov_func=<pymc3.gp.cov.Constant object>)¶
Latent Gaussian process.
The gp.Latent class is a direct implementation of a GP. No additive noise is assumed. It is called “Latent” because the underlying function values are treated as latent variables. It has a prior method and a conditional method. Given a mean and covariance function the function \(f(x)\) is modeled as,
\[f(x) \sim \mathcal{GP}\left(\mu(x), k(x, x')\right)\]Use the prior and conditional methods to actually construct random variables representing the unknown, or latent, function whose distribution is the GP prior or GP conditional. This GP implementation can be used to implement regression on data that is not normally distributed. For more information on the prior and conditional methods, see their docstrings.
 Parameters
 cov_func: None, 2D array, or instance of Covariance
The covariance function. Defaults to zero.
 mean_func: None, instance of Mean
The mean function. Defaults to zero.
Examples
# A one dimensional column vector of inputs. X = np.linspace(0, 1, 10)[:, None] with pm.Model() as model: # Specify the covariance function. cov_func = pm.gp.cov.ExpQuad(1, ls=0.1) # Specify the GP. The default mean function is `Zero`. gp = pm.gp.Latent(cov_func=cov_func) # Place a GP prior over the function f. f = gp.prior("f", X=X) ... # After fitting or sampling, specify the distribution # at new points with .conditional Xnew = np.linspace(1, 2, 50)[:, None] with model: fcond = gp.conditional("fcond", Xnew=Xnew)
 conditional(name, Xnew, given=None, **kwargs)¶
Returns the conditional distribution evaluated over new input locations Xnew.
Given a set of function values f that the GP prior was over, the conditional distribution over a set of new points, f_* is
\[f_* \mid f, X, X_* \sim \mathcal{GP}\left( K(X_*, X) K(X, X)^{1} f \,, K(X_*, X_*)  K(X_*, X) K(X, X)^{1} K(X, X_*) \right)\] Parameters
 name: string
Name of the random variable
 Xnew: arraylike
Function input values.
 given: dict
Can optionally take as key value pairs: X, y, noise, and gp. See the section in the documentation on additive GP models in PyMC3 for more information.
 **kwargs
Extra keyword arguments that are passed to MvNormal distribution constructor.
 prior(name, X, reparameterize=True, **kwargs)¶
Returns the GP prior distribution evaluated over the input locations X.
This is the prior probability over the space of functions described by its mean and covariance function.
\[f \mid X \sim \text{MvNormal}\left( \mu(X), k(X, X') \right)\] Parameters
 name: string
Name of the random variable
 X: arraylike
Function input values.
 reparameterize: bool
Reparameterize the distribution by rotating the random variable by the Cholesky factor of the covariance matrix.
 **kwargs
Extra keyword arguments that are passed to distribution constructor.
 class pymc3.gp.gp.LatentKron(mean_func=<pymc3.gp.mean.Zero object>, cov_funcs=<pymc3.gp.cov.Constant object>)¶
Latent Gaussian process whose covariance is a tensor product kernel.
The gp.LatentKron class is a direct implementation of a GP with a Kronecker structured covariance, without reference to any noise or specific likelihood. The GP is constructed with the prior method, and the conditional GP over new input locations is constructed with the conditional method. conditional and method. For more information on these methods, see their docstrings. This GP implementation can be used to model a Gaussian process whose inputs cover evenly spaced grids on more than one dimension. LatentKron is relies on the KroneckerNormal distribution, see its docstring for more information.
 Parameters
 cov_funcs: list of Covariance objects
The covariance functions that compose the tensor (Kronecker) product. Defaults to [zero].
 mean_func: None, instance of Mean
The mean function. Defaults to zero.
Examples
# One dimensional column vectors of inputs X1 = np.linspace(0, 1, 10)[:, None] X2 = np.linspace(0, 2, 5)[:, None] Xs = [X1, X2] with pm.Model() as model: # Specify the covariance functions for each Xi cov_func1 = pm.gp.cov.ExpQuad(1, ls=0.1) # Must accept X1 without error cov_func2 = pm.gp.cov.ExpQuad(1, ls=0.3) # Must accept X2 without error # Specify the GP. The default mean function is `Zero`. gp = pm.gp.LatentKron(cov_funcs=[cov_func1, cov_func2]) # ... # After fitting or sampling, specify the distribution # at new points with .conditional # Xnew need not be on a full grid Xnew1 = np.linspace(1, 2, 10)[:, None] Xnew2 = np.linspace(0, 3, 10)[:, None] Xnew = np.concatenate((Xnew1, Xnew2), axis=1) # Not full grid, works Xnew = pm.math.cartesian(Xnew1, Xnew2) # Full grid, also works with model: fcond = gp.conditional("fcond", Xnew=Xnew)
 conditional(name, Xnew, **kwargs)¶
Returns the conditional distribution evaluated over new input locations Xnew.
Xnew will be split by columns and fed to the relevant covariance functions based on their input_dim. For example, if cov_func1, cov_func2, and cov_func3 have input_dim of 2, 1, and 4, respectively, then Xnew must have 7 columns and a covariance between the prediction points
cov_func(Xnew) = cov_func1(Xnew[:, :2]) * cov_func1(Xnew[:, 2:3]) * cov_func1(Xnew[:, 3:])
The distribution returned by conditional does not have a Kronecker structure regardless of whether the input points lie on a full grid. Therefore, Xnew does not need to have grid structure.
 Parameters
 name: string
Name of the random variable
 Xnew: arraylike
Function input values. If onedimensional, must be a column vector with shape (n, 1).
 **kwargs
Extra keyword arguments that are passed to MvNormal distribution constructor.
 prior(name, Xs, **kwargs)¶
Returns the prior distribution evaluated over the input locations Xs.
 Parameters
 name: string
Name of the random variable
 Xs: list of arraylike
Function input values for each covariance function. Each entry must be passable to its respective covariance without error. The total covariance function is measured on the full grid cartesian(*Xs).
 **kwargs
Extra keyword arguments that are passed to the KroneckerNormal distribution constructor.
 class pymc3.gp.gp.Marginal(mean_func=<pymc3.gp.mean.Zero object>, cov_func=<pymc3.gp.cov.Constant object>)¶
Marginal Gaussian process.
The gp.Marginal class is an implementation of the sum of a GP prior and additive noise. It has marginal_likelihood, conditional and predict methods. This GP implementation can be used to implement regression on data that is normally distributed. For more information on the prior and conditional methods, see their docstrings.
 Parameters
 cov_func: None, 2D array, or instance of Covariance
The covariance function. Defaults to zero.
 mean_func: None, instance of Mean
The mean function. Defaults to zero.
Examples
# A one dimensional column vector of inputs. X = np.linspace(0, 1, 10)[:, None] with pm.Model() as model: # Specify the covariance function. cov_func = pm.gp.cov.ExpQuad(1, ls=0.1) # Specify the GP. The default mean function is `Zero`. gp = pm.gp.Marginal(cov_func=cov_func) # Place a GP prior over the function f. sigma = pm.HalfCauchy("sigma", beta=3) y_ = gp.marginal_likelihood("y", X=X, y=y, noise=sigma) ... # After fitting or sampling, specify the distribution # at new points with .conditional Xnew = np.linspace(1, 2, 50)[:, None] with model: fcond = gp.conditional("fcond", Xnew=Xnew)
 conditional(name, Xnew, pred_noise=False, given=None, **kwargs)¶
Returns the conditional distribution evaluated over new input locations Xnew.
Given a set of function values f that the GP prior was over, the conditional distribution over a set of new points, f_* is:
\[f_* \mid f, X, X_* \sim \mathcal{GP}\left( K(X_*, X) [K(X, X) + K_{n}(X, X)]^{1} f \,, K(X_*, X_*)  K(X_*, X) [K(X, X) + K_{n}(X, X)]^{1} K(X, X_*) \right)\] Parameters
 name: string
Name of the random variable
 Xnew: arraylike
Function input values. If onedimensional, must be a column vector with shape (n, 1).
 pred_noise: bool
Whether or not observation noise is included in the conditional. Default is False.
 given: dict
Can optionally take as key value pairs: X, y, noise, and gp. See the section in the documentation on additive GP models in PyMC3 for more information.
 **kwargs
Extra keyword arguments that are passed to MvNormal distribution constructor.
 marginal_likelihood(name, X, y, noise, is_observed=True, **kwargs)¶
Returns the marginal likelihood distribution, given the input locations X and the data y.
This is integral over the product of the GP prior and a normal likelihood.
\[y \mid X,\theta \sim \int p(y \mid f,\, X,\, \theta) \, p(f \mid X,\, \theta) \, df\] Parameters
 name: string
Name of the random variable
 X: arraylike
Function input values. If onedimensional, must be a column vector with shape (n, 1).
 y: arraylike
Data that is the sum of the function with the GP prior and Gaussian noise. Must have shape (n, ).
 noise: scalar, Variable, or Covariance
Standard deviation of the Gaussian noise. Can also be a Covariance for nonwhite noise.
 is_observed: bool
Whether to set y as an observed variable in the model. Default is True.
 **kwargs
Extra keyword arguments that are passed to MvNormal distribution constructor.
 predict(Xnew, point=None, diag=False, pred_noise=False, given=None)¶
Return the mean vector and covariance matrix of the conditional distribution as numpy arrays, given a point, such as the MAP estimate or a sample from a trace.
 Parameters
 Xnew: arraylike
Function input values. If onedimensional, must be a column vector with shape (n, 1).
 point: pymc3.model.Point
A specific point to condition on.
 diag: bool
If True, return the diagonal instead of the full covariance matrix. Default is False.
 pred_noise: bool
Whether or not observation noise is included in the conditional. Default is False.
 given: dict
Same as conditional method.
 predictt(Xnew, diag=False, pred_noise=False, given=None)¶
Return the mean vector and covariance matrix of the conditional distribution as symbolic variables.
 Parameters
 Xnew: arraylike
Function input values. If onedimensional, must be a column vector with shape (n, 1).
 diag: bool
If True, return the diagonal instead of the full covariance matrix. Default is False.
 pred_noise: bool
Whether or not observation noise is included in the conditional. Default is False.
 given: dict
Same as conditional method.
 class pymc3.gp.gp.MarginalKron(mean_func=<pymc3.gp.mean.Zero object>, cov_funcs=<pymc3.gp.cov.Constant object>)¶
Marginal Gaussian process whose covariance is a tensor product kernel.
The gp.MarginalKron class is an implementation of the sum of a Kronecker GP prior and additive white noise. It has marginal_likelihood, conditional and predict methods. This GP implementation can be used to efficiently implement regression on data that are normally distributed with a tensor product kernel and are measured on a full grid of inputs: cartesian(*Xs). MarginalKron is based on the KroneckerNormal distribution, see its docstring for more information. For more information on the prior and conditional methods, see their docstrings.
 Parameters
 cov_funcs: list of Covariance objects
The covariance functions that compose the tensor (Kronecker) product. Defaults to [zero].
 mean_func: None, instance of Mean
The mean function. Defaults to zero.
Examples
# One dimensional column vectors of inputs X1 = np.linspace(0, 1, 10)[:, None] X2 = np.linspace(0, 2, 5)[:, None] Xs = [X1, X2] y = np.random.randn(len(X1)*len(X2)) # toy data with pm.Model() as model: # Specify the covariance functions for each Xi cov_func1 = pm.gp.cov.ExpQuad(1, ls=0.1) # Must accept X1 without error cov_func2 = pm.gp.cov.ExpQuad(1, ls=0.3) # Must accept X2 without error # Specify the GP. The default mean function is `Zero`. gp = pm.gp.MarginalKron(cov_funcs=[cov_func1, cov_func2]) # Place a GP prior over the function f. sigma = pm.HalfCauchy("sigma", beta=3) y_ = gp.marginal_likelihood("y", Xs=Xs, y=y, sigma=sigma) # ... # After fitting or sampling, specify the distribution # at new points with .conditional # Xnew need not be on a full grid Xnew1 = np.linspace(1, 2, 10)[:, None] Xnew2 = np.linspace(0, 3, 10)[:, None] Xnew = np.concatenate((Xnew1, Xnew2), axis=1) # Not full grid, works Xnew = pm.math.cartesian(Xnew1, Xnew2) # Full grid, also works with model: fcond = gp.conditional("fcond", Xnew=Xnew)
 conditional(name, Xnew, pred_noise=False, **kwargs)¶
Returns the conditional distribution evaluated over new input locations Xnew, just as in Marginal.
Xnew will be split by columns and fed to the relevant covariance functions based on their input_dim. For example, if cov_func1, cov_func2, and cov_func3 have input_dim of 2, 1, and 4, respectively, then Xnew must have 7 columns and a covariance between the prediction points
cov_func(Xnew) = cov_func1(Xnew[:, :2]) * cov_func1(Xnew[:, 2:3]) * cov_func1(Xnew[:, 3:])
The distribution returned by conditional does not have a Kronecker structure regardless of whether the input points lie on a full grid. Therefore, Xnew does not need to have grid structure.
 Parameters
 name: string
Name of the random variable
 Xnew: arraylike
Function input values. If onedimensional, must be a column vector with shape (n, 1).
 pred_noise: bool
Whether or not observation noise is included in the conditional. Default is False.
 **kwargs
Extra keyword arguments that are passed to MvNormal distribution constructor.
 marginal_likelihood(name, Xs, y, sigma, is_observed=True, **kwargs)¶
Returns the marginal likelihood distribution, given the input locations cartesian(*Xs) and the data y.
 Parameters
 name: string
Name of the random variable
 Xs: list of arraylike
Function input values for each covariance function. Each entry must be passable to its respective covariance without error. The total covariance function is measured on the full grid cartesian(*Xs).
 y: arraylike
Data that is the sum of the function with the GP prior and Gaussian noise. Must have shape (n, ).
 sigma: scalar, Variable
Standard deviation of the white Gaussian noise.
 is_observed: bool
Whether to set y as an observed variable in the model. Default is True.
 **kwargs
Extra keyword arguments that are passed to KroneckerNormal distribution constructor.
 predict(Xnew, point=None, diag=False, pred_noise=False)¶
Return the mean vector and covariance matrix of the conditional distribution as numpy arrays, given a point, such as the MAP estimate or a sample from a trace.
 Parameters
 Xnew: arraylike
Function input values. If onedimensional, must be a column vector with shape (n, 1).
 point: pymc3.model.Point
A specific point to condition on.
 diag: bool
If True, return the diagonal instead of the full covariance matrix. Default is False.
 pred_noise: bool
Whether or not observation noise is included in the conditional. Default is False.
 predictt(Xnew, diag=False, pred_noise=False)¶
Return the mean vector and covariance matrix of the conditional distribution as symbolic variables.
 Parameters
 Xnew: arraylike
Function input values. If onedimensional, must be a column vector with shape (n, 1).
 diag: bool
If True, return the diagonal instead of the full covariance matrix. Default is False.
 pred_noise: bool
Whether or not observation noise is included in the conditional. Default is False.
 class pymc3.gp.gp.MarginalSparse(mean_func=<pymc3.gp.mean.Zero object>, cov_func=<pymc3.gp.cov.Constant object>, approx='FITC')¶
Approximate marginal Gaussian process.
The gp.MarginalSparse class is an implementation of the sum of a GP prior and additive noise. It has marginal_likelihood, conditional and predict methods. This GP implementation can be used to implement regression on data that is normally distributed. The available approximations are:
DTC: Deterministic Training Conditional
FITC: Fully independent Training Conditional
VFE: Variational Free Energy
 Parameters
 cov_func: None, 2D array, or instance of Covariance
The covariance function. Defaults to zero.
 mean_func: None, instance of Mean
The mean function. Defaults to zero.
 approx: string
The approximation to use. Must be one of VFE, FITC or DTC.
References
QuinoneroCandela, J., and Rasmussen, C. (2005). A Unifying View of Sparse Approximate Gaussian Process Regression.
Titsias, M. (2009). Variational Learning of Inducing Variables in Sparse Gaussian Processes.
Examples
# A one dimensional column vector of inputs. X = np.linspace(0, 1, 10)[:, None] # A smaller set of inducing inputs Xu = np.linspace(0, 1, 5)[:, None] with pm.Model() as model: # Specify the covariance function. cov_func = pm.gp.cov.ExpQuad(1, ls=0.1) # Specify the GP. The default mean function is `Zero`. gp = pm.gp.MarginalSparse(cov_func=cov_func, approx="FITC") # Place a GP prior over the function f. sigma = pm.HalfCauchy("sigma", beta=3) y_ = gp.marginal_likelihood("y", X=X, Xu=Xu, y=y, sigma=sigma) ... # After fitting or sampling, specify the distribution # at new points with .conditional Xnew = np.linspace(1, 2, 50)[:, None] with model: fcond = gp.conditional("fcond", Xnew=Xnew)
 conditional(name, Xnew, pred_noise=False, given=None, **kwargs)¶
Returns the approximate conditional distribution of the GP evaluated over new input locations Xnew.
 Parameters
 name: string
Name of the random variable
 Xnew: arraylike
Function input values. If onedimensional, must be a column vector with shape (n, 1).
 pred_noise: bool
Whether or not observation noise is included in the conditional. Default is False.
 given: dict
Can optionally take as key value pairs: X, Xu, y, noise, and gp. See the section in the documentation on additive GP models in PyMC3 for more information.
 **kwargs
Extra keyword arguments that are passed to MvNormal distribution constructor.
 marginal_likelihood(name, X, Xu, y, noise=None, is_observed=True, **kwargs)¶
Returns the approximate marginal likelihood distribution, given the input locations X, inducing point locations Xu, data y, and white noise standard deviations sigma.
 Parameters
 name: string
Name of the random variable
 X: arraylike
Function input values. If onedimensional, must be a column vector with shape (n, 1).
 Xu: arraylike
The inducing points. Must have the same number of columns as X.
 y: arraylike
Data that is the sum of the function with the GP prior and Gaussian noise. Must have shape (n, ).
 noise: scalar, Variable
Standard deviation of the Gaussian noise.
 is_observed: bool
Whether to set y as an observed variable in the model. Default is True.
 **kwargs
Extra keyword arguments that are passed to MvNormal distribution constructor.
 class pymc3.gp.gp.TP(mean_func=<pymc3.gp.mean.Zero object>, cov_func=<pymc3.gp.cov.Constant object>, nu=None)¶
Student’s T process prior.
The usage is nearly identical to that of gp.Latent. The differences are that it must be initialized with a degrees of freedom parameter, and TP is not additive. Given a mean and covariance function, and a degrees of freedom parameter, the function \(f(x)\) is modeled as,
\[\begin{split}f(X) \sim \mathcal{TP}\left( \mu(X), k(X, X'), \\nu \\right)\end{split}\] Parameters
 cov_funcNone, 2D array, or instance of Covariance
The covariance function. Defaults to zero.
 mean_funcNone, instance of Mean
The mean function. Defaults to zero.
 nufloat
The degrees of freedom
References
Shah, A., Wilson, A. G., and Ghahramani, Z. (2014). Studentt Processes as Alternatives to Gaussian Processes. arXiv preprint arXiv:1402.4306.
 conditional(name, Xnew, **kwargs)¶
Returns the conditional distribution evaluated over new input locations Xnew.
Given a set of function values f that the TP prior was over, the conditional distribution over a set of new points, f_* is
 Parameters
 name: string
Name of the random variable
 Xnew: arraylike
Function input values.
 **kwargs
Extra keyword arguments that are passed to MvNormal distribution constructor.
 prior(name, X, reparameterize=True, **kwargs)¶
Returns the TP prior distribution evaluated over the input locations X.
This is the prior probability over the space of functions described by its mean and covariance function.
 Parameters
 name: string
Name of the random variable
 X: arraylike
Function input values.
 reparameterize: bool
Reparameterize the distribution by rotating the random variable by the Cholesky factor of the covariance matrix.
 **kwargs
Extra keyword arguments that are passed to distribution constructor.