Chapter 12.03: (Massively) Multilingual Transformers
As we have previously seen, transformers are the working horse for modern NLP. In this section we learn how transformers are adapted to work in multilingual settings and discuss typical issues when training those. Furthermore we look at zero-shot cross lingual transfer capabilities.