Topic 1: Introduction

In this section, we introduces the relationship of DL and ML, give basic intro about feature learning, and discuss the use-cases and data types for DL methods.

In this section we explain the graphical representation of a single neuron and describe affine transformations and non-linear activation functions. Moreover, we talk about the hypothesis spaces of a single neuron and name some typical loss functions.

Example problem a single neuron can not solve but a single hidden layer net can!

We introduce architecture of single hidden layer neural networks and discuss the advantage of hidden layers. Then, we explain the typical (non-linear) activation functions.

In this section, we discuss a neural network architectures for multi-class classification, softmax activation function as well as the Softmax loss.

Architectures of deep neural networks and deep neural networks as chained functions are the learning goal of this part.

In this section we learn about compact representation of neural network, vector notation for neuron layers, vector and matrix notation of bias and weight parameters.

Universal approximation theorem for one-hidden-layer neural networks and pros and cons of a low approximation error are the learning goal of this section.

We overview history of DL development.