Chapter 1: ML Basics

In this nutshell chunk, we dive into the foundational principles of Machine Learning.

As subtopic of artificial intelligence, machine learning is a mathematically well-defined discipline and usually constructs predictive or decision models from data, instead of explicitly programming them. In this section, you will see some typical examples of where machine learning is applied and the main directions of machine learning.

In this section we explain the basic structure of tabular data used in machine learning. We will differentiate targets from features, talk about labeled and unlabeled data and introduce the concept of the data generating process.

The tasks of supervised learning can roughly be divided in two categories: regression (for continuous outcome) and classification (for categorical outcome). We will present some examples.

We introduce models as functional hypotheses about the mapping from feature to target space that allow us to make predictions by computing a function of the input data. Frequently in machine learning, models are understood to be parameterized curves, which is illustrated by several examples.

Roughly speaking, learners (endowed with a specific hyperparameter configuration) take training data and return a model.

In order to find good solutions we need a concept to evaluate and compare models. To this end, the concepts of loss function, risk and empirical risk minimization are introduced.

In this section we study parameter optimization as computational solution to machine learning problems. We address pitfalls in non-convex optimization problems and introduce the fundamental concept of gradient descent.

Nearly all supervised learning algorithms can be described in terms of three components: 1) hypothesis space, 2) risk, and 3) optimization. In this section, we explain how these components interact and why this is a very useful concept for many supervised learning approaches.