Chapter 13.05: Cross-Entropy and KL

We introduce cross-entropy as a further information-theoretic concept and discuss the connection between entropy, cross-entropy, and Kullback-Leibler divergence.

Lecture video

Lecture slides