Why do you need base training?

[hdnminh]

Hi Matthias
@mperkonigg,

I hope you are doing well with your plans/works

Why do you need base training when you can train on dataset A and evaluate on testing set itself? And then you can continually train on datasets B, C, and D.
I mean that you could not have needed base training but you still did it. Can you explain why?

Thank you in advance. I am looking forward to your response soon.

Best,
Minh

[mperkonigg]

Hi @hdnminh !

Sorry for the late reply.

Base training is needed to establish a model that works well on one domain (dataset A) for the given task. This is a common setup in continual learning tasks. Starting from a randomly initialized model, and training with only continual learning would lead to poor performance because the model has received to little weight updates.

Besides, having an initial dataset collected for a single domain (dataset A) for standard, batch-wise training is a realistic scenario. The issue we want to tackle with our approach is the dataset shift that occurs after a model is fully trained on a single domain.

I hope this answers your question. Feel free to reach out if you have any more questions!

Best,
Matthias