Chapter 15.09: Geometric Analysis of L2 Regularization and Weight Decay

In this section, we provide a geometric understanding of \(L2\) regularization, showing how parameters are shrunk according to the eigenvalues of the Hessian of empirical risk, and discuss its correspondence to weight decay.

Lecture video

Lecture slides

Download »slides-regu-geom-l2-wdecay.pdf«

• « Chapter 15.08: Regularization and Bayesian Priors
• Chapter 15.10: Geometric Analysis of L1 Regularization »