Appendix

Look-up slides for important ML learners

• I2ML :: BASICS

• I2ML :: EVALUATION & TUNING

Errata in the slides shown in the videos

• Chapter 1.4 (Models & Parameters) - slide 5/10: d-dimensional vector, not p-dimensional
• Chapter 2.1 (Regression losses): Slide 1/5 sign in bullet point 4
• Chapter 2.2 (Deep Dive OLS): Slide 2/2 last lines in left column
• Chapter 3.6 (Naive Bayes): Slide 3/6: Missing exponents in figure
• Chapter 4.3 (Simple Measures for Classification) - slide 6/9: Error in cost matrix
• Chapter 4.4 (Perfomance Evaluation: Test Error) - slide 8/13: The variance of MSE decreases with test set size, not the mean of MSE
• Chapter 4.7 (Classification measures): Slide 6/9 cost computation
• Chapter 6.2 (CART: Growing a Tree) - slide 5/8: Wrong plot is displayed in video (axis wrong, points missing)
• Chapter 11.6 (0-1 Loss): Slides 2/5 and 4/5 Errors in notation of conditional probability inside of expectation
• Chapter 11.7 (Bernoulli Loss): Slides 9/10 and 10/10 Errors in Bernoulli Loss and Entropy Splitting Criterion
• Chapter 11.12 (MLE2): Slide 2/5 wrong negative sign in NLL equation
• Chapter 12.2 (Softmax): Slide 2/9 [0,1]^g instead of R^g
• Chapter 13.1 (Entropy I): Slide 4,6,8/10 changed entropy calculation from nats to bits
• Chapter 13.2 (Entropy II): Slide 1/7 corrected plot for entropy of Bernoulli distribution
• Chapter 13.5 (CE-KLD): Slide 6/7 typo in formula (1)
• Chapter 13.6: Slide 2/7 typos in formula and bullet point 3
• Chapter 13.7: Slide 4/14 switched x and y in the proposition regarding zero conditional entropy
• Chapter 13.7: Slide 14/14 added missing 0.5 factor in the entropy of the multivariate Gaussian
• Chapter 13.7: Slide 14/14 added parentheses to make log less ambiguous
• Chapter 15.2 (Ridge Regression): Slide 4/10 clarified meaning of green dot in plot in comment
• Chapter 15.8 (Bayesian Priors): Slide 5/5 renew plot and add comment on ybar
• Chapter 17.1: Slide 6/11 Typo in f
• Chapter 17.3: Slide 2/6 Show how to get phi(x) explicitly, and the last component of phi(x) should be b instead of sqrt(b).
• Chapter 17.3: Slide 2/7 Assume gamma = 1 to avoid conflict with geometric distances
• Chapter 18.1: Slide 5/7: Typo in Monomials; it should be (d+p-1 over d) and ( (d+p) over d) -1
• Chapter 19.2: Slide 1/15: R_emp formula contains x_{i}, not x.
• Chapter 19.5: Slide 3/11: bullet point 1 added “if”, bullet point 2 changed “decrease” to “increase”
• Chapter 20.1: Slide 6/12: fix error in step 6 and clarify that b-hat is a hard label classifier
• Chapter 20.1: Slides 7 and 8/12: improved plot and text
• Chapter 20.2: Slide 14/15: error in step 1 of the algorithm
• Chapter 21.4: Slide 2/5: increased number of boosting iterations from 1 to 300 in plot
• Chapter 21.4: Slide 5/5: add to for loops over i=1,..,n samples and initialize empty data set
• Chapter 13.1: Slide 10/10: fixed incorrect derivative
• Chapter 18.08: Slide 6/8: the slide set is correct, i.e., the given percentage is used to “dropout” / ignore the learners. (This is currently incorrectly stated in the video)

Other ML courses

• Deep Learning for NLP (DL4NLP)
• Introduction to Deep Learning (I2DL)

Data sets used in the lecture

• Bikeshare

• Boston Housing

We are aware of the ethical issues regarding this data set (more information) and will replace it in upcoming revisions of the material.

• Circle

• German Credit

• Glass

• Ionosphere

• Iris

We are aware of the ethical issues regarding this data set (more information) and will replace it in upcoming revisions of the material.

• MNIST

• Sonar

• Spam Classification

• Spirals

• Titanic Survival

• Waveform