Chapter 20: Gaussian Processes
This chapter introduces Gaussian processes as a model class. Gaussian processes are non-parametric approaches with ubiquitous application that model entire distributions in function space.
-
Chapter 20.01: The Bayesian Linear Model
We begin by reviewing the Bayesian formulation of a linear model and show that instead of point estimates for parameters and predictions, we obtain an entire posterior and predictive distribution.
-
Chapter 20.02: Gaussian Processes
In this section, we introduce the basic idea behind Gaussian processes. We move from weight to function space and build some intuition on distributions over functions, discuss GPs’ marginalization property, derive GP priors, and interpret GPs as indexed families.
-
Chapter 20.03: Covariance Functions for GPs
In this section, we discuss the role of covariance functions in GPs and introduce the most common choices.
-
Chapter 20.04: Gaussian Process Prediction
In this section, we show how to derive the posterior process and discuss further properties of GPs as well as noisy GPs.
-
Chapter 20.05: Gaussian Process Training
In this section, we show how Gaussian processes are actually trained using maximum likelihood estimation and exploiting the fact that we can learn covariance functions’ hyperparameters on the fly.