Chapter 15.09: Geometry of L2 Regularization

In this section, we provide a geometric understanding of \(L2\) regularization, showing how parameters are shrunk according to the eigenvalues of the Hessian of empirical risk, and discuss its correspondence to weight decay.

Lecture video

Lecture slides

Download »slides-regu-geom-l2.pdf«

• « Chapter 15.08: Bayesian Priors
• Chapter 15.10: Geometry of L1 Regularization »