• sequential data processing
• recurrent neural networks
• continual learning

Continual Learning (CL) is the process of learning new things on top of what has already been learned. Most of the existing works focus on CL for Computer Vision or Reinforcement Learning, but very few experiments are available on CL for sequential data processing, which is a fundamental area of Machine Learning (ML), with very important applications in Natural Language Processing, robotics, bioinformatics and many other fields.
This thesis explores the problem of learning sequential data continuously, without forgetting previously acquired knowledge (hence preventing catastrophic forgetting phenomenon) while at the same time adapting to new input distributions. The approach developed in this thesis is based on Recurrent Neural Networks (RNNs), a model widely used in ML. The novel contribution introduced in this work extend both a traditional recurrent model like the Long-Short Term Memory (LSTM) and a recently developed recurrent model like the Linear Memory Network (LMN), augmenting them by dynamically adding new recurrent modules on top of the existing ones and by connecting the resulting architecture via adaptive parameters.
The proposed approach is validated against recurrent baseline models like plain LSTM and LMN on available datasets for sequential data processing.