Introduction to Machine Learning (I2ML) | Chapter 08: Autoencoders

This chapter introduces unsupervised learning with a focus on autoencoders (AEs), which learn compact representations of data without labeled outputs. It explains the structure of AEs, including encoder-decoder frameworks and their use in dimensionality reduction and feature extraction. The chapter also explores regularized variants such as overcomplete, sparse, denoising, and contractive AEs, highlighting their unique roles in improving representation quality. Finally, it covers convolutional AEs for image data and manifold learning concepts.

Chapter 08.01: Unsupervised Learning
This subchapter provides an overview of unsupervised learning, focusing on discovering patterns and structures within unlabeled data. Key topics include clustering, dimensionality reduction, feature extraction, and generative modeling, each demonstrating different ways to learn and represent underlying data structures.
Chapter 08.02: Autoencoders
This subchapter covers the task and structure of AEs, which compress data into a lower-dimensional latent space and reconstruct it. In addition, we focus on undercomplete AEs, enforcing a “bottleneck” to focus on essential features. Linear undercomplete AEs with L2-reconstruction error approximate PCA by identifying principal components, while nonlinear AEs extend this capability to capture complex data patterns.
Chapter 08.03: Regularized Autoencoders
In this subsection, we explain overcomplete AEs, sparse AEs, denoising AEs and contractive AEs.
Chapter 08.04: Specific Autoencoders and Applications
This subchapter introduces convolutional autoencoders (ConvAEs), which utilize convolutional and transpose convolutional layers for processing image data. Furthermore, some practical applications e.g. the denoising of medical images or image compression are briefly presented.
Chapter 08.05: Manifold Learning
In this subchapter we explore the concept of manifold learning, focusing on the manifold hypothesis. In addition, we present AEs as tools for learning such manifolds.