We discuss the concept of robustness, analytical and functional properties of loss functions and how they may influence the convergence of optimizers.
