Deep Learning for NLP (DL4NLP)
This website offers two open and free introductory courses on deep learning algorithms and popular architectures for contemporary Natural Language Processing (NLP). The courses are constructed holistically and as self-contained as possible, in order to cover all of the basics required for understanding current research. Further, we discuss most of the relevant state-of-the-art architectures and application areas, which simultaneously means that we will basically never be able to cover everything. We continuously develop this course and add further topics/architectures over the course of time.
One general, important goal of the course - on top of clearly explaining the most popular ML algorithms for NLP - is to enable graduate-level students to comprehend the different streamlines of ongoing research in the field and also to critically reflect on them. Further, it is an important goal to provide practical advice on how to use the presented architectures in practice, i.e. teaching the required programming skills needed to work on one’s own practical application.
The course can be taken as a graduate-level course for Master’s level students, both at the University of Munich (LMU) and at the University of Vienna. For the exact details, please refer to the corresponding Moodle pages at either of the two universities.
The course material is developed in a public GitHub repository: https://github.com/slds-lmu/lecture_dl4nlp, where you can also find the changelog for the material.
- Chapters (DL4NLP 1)
- Chapter 00: Machine Learning Basics
- Chapter 01: Introduction to the course
- Chapter 02: Deep Learning Basics
- Chapter 03: Transformer
- Chapter 04: BERT
- Chapter 05: Post-BERT Era
- Chapter 06: Generative Pre-Trained Transformers
- Chapter 07: Decoding Strategies
- Chapter 08: Large Language Models (LLMs)
- Chapter 09: Reinforcement Learning from Human Feedback (RLHF)
- Chapter 10: Training Large Language Models
- Chapters (DL4NLP 2)
- Chapter 11: Recap of the Transformer
- Chapter 12: Tokenization
- Chapter 13: Architecture Variants
- Chapter 14: Advanced LLM Inference
- Chapter 15: Pre-Training
- Chapter 16: Fine-Tuning
- Chapter 17: Post-Training
- Chapter 18: Reasoning and Agents
- Chapter 19: Evaluation and Benchmarking
- Chapter 20: Mechanistic Interpretability
- Chapter 21: Alignment, Biases, and Safety