logo SBA

ETD

Archivio digitale delle tesi discusse presso l’Università di Pisa

Tesi etd-11122023-104119


Tipo di tesi
Tesi di laurea magistrale
Autore
RISTORI, ALESSANDRO
URN
etd-11122023-104119
Titolo
Continual Learning for Non-Autoregressive Neural Machine Translation
Dipartimento
INFORMATICA
Corso di studi
INFORMATICA
Relatori
relatore Prof. Bacciu, Davide
correlatore Dott. Resta, Michele
Parole chiave
  • continual learning
  • continual neural machine translation
  • neural machine translation
  • non-autoregressive neural machine translation
Data inizio appello
01/12/2023
Consultabilità
Tesi non consultabile
Riassunto
This work aims to investigate the capabilities and performance of Non-Autoregressive Neural Machine Translation models in a multilingual and continual setting by comparing them to a standard autoregressive baseline. We decided to conduct our experiments with four high resource languages such as English, German, French and Spanish in an English-centric scenario (translating from and to English).

Firstly, we introduced the main concepts of Neural Machine Translation and the peculiarities of its Non-Autoregressive variants and we went on to describe in more details the two models (named CMLM and GLAT) under our inspection.

Following this, we tested how Non-Autoregressive models compare in a basic multilingual scenario against an Autoregressive baseline by evaluating their performance on two famous benchmarks.

Afterwards, we employed an Incremental Language Learning setting in which the models are trained on a single language pair (i.e.: English-German) at each subsequent experience. In this way we noticed the effects of Catastrophic Forgetting and, finally, we tried to mitigate it by utilizing the Continual Learning strategy of Experience Replay.
File