• continual learning
• continual neural machine translation
• neural machine translation
• non-autoregressive neural machine translation

This work aims to investigate the capabilities and performance of Non-Autoregressive Neural Machine Translation models in a multilingual and continual setting by comparing them to a standard autoregressive baseline. We decided to conduct our experiments with four high resource languages such as English, German, French and Spanish in an English-centric scenario (translating from and to English).

Firstly, we introduced the main concepts of Neural Machine Translation and the peculiarities of its Non-Autoregressive variants and we went on to describe in more details the two models (named CMLM and GLAT) under our inspection.

Following this, we tested how Non-Autoregressive models compare in a basic multilingual scenario against an Autoregressive baseline by evaluating their performance on two famous benchmarks.

Afterwards, we employed an Incremental Language Learning setting in which the models are trained on a single language pair (i.e.: English-German) at each subsequent experience. In this way we noticed the effects of Catastrophic Forgetting and, finally, we tried to mitigate it by utilizing the Continual Learning strategy of Experience Replay.