In this subchapter, we explain challenges in optimization related to activation functions. In addition, we introduce activations for both hidden and output units.
