Implement strategy for assessing the quality of the model during lifelong training

[jcohenadad]

As we are adding more contrasts and re-training the model overtime (see eg: #83, #74, ivadomed/canproco#46), we need to put in place a quality check assessment of model performance shift across various data domains (ie: monitor catastrophic forgetting).

[naga-karthik]

A relevant theory paper I found: Understanding Continual Learning Settings with Data Distribution Drift Analysis -- Essentially describes the theory of data distribution shifts, proposes new concepts in analyzing model/data drifts and some of the existing concepts in lifelong learning that are related to this phenomenon.

Relevant sections: Sections 3.1, 3.2, 4.1, and 6.2

[jcohenadad]

one idea of validation is to compute the CSA variation across contrasts from the test set of the spine generic data