We introduce key properties of loss functions and explore how these influence model assumptions, sensitivity to outliers, and the tractability of training.