In this section we introduce several advanced techniques for optimization of neural network such as learning rate schedules, adaptive learning rates, and batch normalization.
