• centered kernel alignment
• neighborhood distillation
• machine learning
• continual learning
• neural network

Continual learning is a challenging problem in the field of artificial intelligence that involves learning new tasks while preserving previously learned knowledge. In this thesis, we implemented an already existing algorithm, which is called Neighborhood Distillation, and tested it in the field of continual learning.
The Neighborhood Distillation algorithm performs latent distillation by splitting the deep neural network into separate sub-networks called Neighborhoods. We have compared the results obtained with the Neighborhood Distillation algorithm with the state-of-art algorithms in the field of continual learning, such as Learning without Forgetting and Elastic Weight Consolidation. We have also integrated the algorithm with a replay approach.
The experiments test the Neighborhood Distillation algorithm's ability to learn new tasks while preserving previously learned knowledge, its memory efficiency, and its generalization ability. Our results demonstrate that Neighborhood Distillation performs similarly and sometimes better concerning the state-of-art algorithms. Furthermore, our analysis reveals that the performance of Neighborhood Distillation is affected by various factors such as the number of tasks, the complexity of the tasks, and the amount of available memory.
We study the plasticity of the method by measuring the similarity of a learned model at different points in time with the Centered Kernel Alignment.
The results provide evidence of the effectiveness of Neighborhood Distillation. This can lead to future research in the field of continual learning based on the proposed algorithm.