Deep Learning for NLP (DL4NLP)
This website offers an open and free introductory course on deep learning algorithms and popular architectures for contemporary Natural Language Processing (NLP). The course is constructed holistically and as self-contained as possible, in order to cover all of the basics required for understanding current research. Further, we discuss most of the relevant state-of-the-art architectures and application areas, which simultaneously means that we will basically never be able to cover everything. We continuously develop this course and add further topics/architectures over the course of time.
One general, important goal of the course - on top of clearly explaining the most popular ML algorithms for NLP - is to enable graduate-level students to comprehend the different streamlines of ongoing research in the field and also to critically reflect on them. Further, it is an important goal to provide practical advice on how to use the presented architectures in practice, i.e. teaching the required programming skills needed to work on one’s own practical application.
The course can be taken as a graduate-level course for Master’s level students, both at the University of Munich (LMU) and at the University of Vienna. For the exact details, please refer to the corresponding Moodle pages at either of the two universities.
The course material is developed in a public GitHub repository: https://github.com/slds-lmu/lecture_dl4nlp, where you can also find the changelog for the material.
If you love teaching ML and have free resources available, please consider joining the team and email us now:
- University of Munich: matthias@stat.uni-muenchen.de
- University of Vienna: andreas.stephan@univie.ac.at
- Chapter 0: Machine Learning Basics
- Chapter 1: Introduction to the course
- Chapter 2: Deep Learning Basics
- Chapter 3: Transformer
- Chapter 4: BERT
- Chapter 5: Post-BERT Era
- Chapter 6: Post-BERT Era II and using the Transformer
- Chapter 7: Generative Pre-Trained Transformers
- Chapter 8: Decoding Strategies
- Chapter 9: Large Language Models (LLMs)
- Chapter 10: Reinforcement Learning from Human Feedback (RLHF)
- Chapter 11: Training Large Language Models
- Chapter 12: Multilinguality