In this subchapter we introduce a couple of advanced algorithms all building on SGD such as ADAM and AdaGrad.
