ETD

• continual learning
• dataset distillation
• machine learning

Continual Learning refers to a Machine Learning setup where data is presented to the model in a sequential fashion. The main problem faced by Continual Learning is the catastrophic forgetting of existing knowledge when acquiring new information.
We combined a replay-based Continual Learning approach with the Dataset Distillation technique, to mitigate forgetting. Replay approaches store a memory of previous patterns and interleave them with incoming data at training time. Dataset Distillation allows to compress an entire dataset into a small set of informative examples. We used the distilled patterns as replay memory and showed superior performance with respect to traditional replay on three Continual Learning benchmarks.