In this section we study parameter optimization as computational solution to machine learning problems. We address pitfalls in non-convex optimization problems and introduce the fundamental concept of gradient descent.