In this subsection, we summarize several of the most prominent challenges regarding training of deep neural networks such as ill-conditioning, local minima, saddle points, cliffs and exploding gradients.
