Chapter 03.04: Geometric Analysis of L2 Regularization and Weight Decay

In this section, we provide a geometric understanding of \(L2\) regularization, showing how parameters are shrunk according to the eigenvalues of the Hessian of empirical risk, and discuss its correspondence to weight decay. (Material provided by: I2ML/SL lecture. )

Lecture video

Lecture slides

Download »slides-regu-geom-l2.pdf«

• « Chapter 03.03: Regularization in Non-Linear Models
• Chapter 03.05: Early Stopping »