Added Deep SLDA Model implementation

[tyler-hayes]

I added deep_slda to training/strategies and an example script to run it on CORe50 in examples/deep_slda

The model works as expected, however I am a bit unsure about how to integrate the avalanche metrics and logging features into these scripts. Deep SLDA does not perform gradient descent, so it trains a bit differently than the default BaseStrategy

Would someone be able to set some time up to discuss this implementation with me so I can get Deep SLDA integrated with the Avalanche loggers and metrics? Thank you in advance!

@AntonioCarta

[coveralls]

Pull Request Test Coverage Report for Build 723237734

• 0 of 0 changed or added relevant lines in 0 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage remained the same at 76.329%

---

Totals
Change from base Build 721954277:	0.0%
Covered Lines:	6520
Relevant Lines:	8542

---

💛 - Coveralls

[AndreaCossu]

Hi @tyler-hayes ! Thanks for the work. Yes, right now the base strategy it is kind of tied to the whole backpropagation way of training models, so I can see why you cannot implement your strategy following that template. That also prevents you to use the default logging features. We can surely discuss a custom solution (standalone metrics or management of the evaluation plugin within your strategy), but maybe this points to a larger problem, which could be solved with a different base strategy/extension of the current one? @AntonioCarta

[AndreaCossu]

Maybe also overriding the training_epoch callback (and other ones?) from the base strategy could help. This would overwrite the usage of backpropagation and gradient descent with your custom code. Anyway, we have to think about it together to shape some solution.

[AndreaCossu]

Hi @tyler-hayes , I'm currently working on refactoring the strategy.
Could you please tell me what is the expected performance I should achieve on the example you wrote?

[tyler-hayes]

Hi @AndreaCossu, thanks for your help!

I used the following parameter arguments to test the model (CORe50-NC):

{"n_classes": 50, "scenario": "nc", "dataset_dir": "/media/tyler/Data/datasets/core50/", "save_dir": "results/nc", "arch": "resnet18", "feature_size": 512, "shrinkage": 0.0001, "plastic_cov": true, "batch_size": 512}

This resulted in the following set of accuracies on the full test set after each batch:

[0.19227519345370453, 0.2798407898247799, 0.3553322067063951, 0.4337587832429067, 0.5035355332206707, 0.5700435826736636, 0.6325269056301699, 0.7184025615938806, 0.7970737347683002]

Feel free to let me know if you need any additional help! :D

[AndreaCossu]

Great, thanks! That should be enough to reproduce results 😄

[AndreaCossu]

This will be added in #583

[AntonioCarta]

Closing this since we are going to merge the other one.