Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MER Strategy + OnlineSupervisedMetaLearning Template #1227

Merged
merged 8 commits into from Nov 29, 2022

Conversation

HamedHemati
Copy link
Collaborator

This PR adds:

  • OnlineSupervisedMetaLearningTemplate: added to the list of common templates
  • MER Strategy: an online meta-learning-based strategy (https://arxiv.org/abs/1810.11910)

The algorithm is adapted from version 1 of the algorithm in the official repository:
https://github.com/mattriemer/MER/blob/master/model/meralg1.py

Unfortunately, the results are only available for MNIST and Omniglot in the original paper, and the rest are RL experiments. In the first run, I tested it SimpleMLP from Avalanche on PMNIST, and the results were a little bit higher compared to what is reported in the paper (~0.76 vs. ~0.73), which I guess should be related to the higher capacity of the default MLP model in Avalanche. I can also run it for the original architecture, but since the algorithm is super slow due to the multiple levels of meta-updates in each iteration, I am going to add them later.

@coveralls
Copy link

coveralls commented Nov 26, 2022

Pull Request Test Coverage Report for Build 3569271524

  • 12 of 17 (70.59%) changed or added relevant lines in 4 files are covered.
  • 4 unchanged lines in 2 files lost coverage.
  • Overall coverage increased (+0.003%) to 73.734%

Changes Missing Coverage Covered Lines Changed/Added Lines %
avalanche/training/templates/common_templates.py 8 9 88.89%
avalanche/logging/interactive_logging.py 1 5 20.0%
Files with Coverage Reduction New Missed Lines %
avalanche/benchmarks/scenarios/classification_scenario.py 2 86.31%
avalanche/training/templates/base.py 2 91.06%
Totals Coverage Status
Change from base Build 3541326024: 0.003%
Covered Lines: 13646
Relevant Lines: 18507

💛 - Coveralls


"""

PLUGIN_CLASS = BaseSGDPlugin
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be SupervisedPlugin

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed for all!

@AntonioCarta
Copy link
Collaborator

For the reproducibility we can discuss this on the continual-learning-baselines repo. If we can have the original architecture it's better.

@AntonioCarta AntonioCarta merged commit 6ccec93 into ContinualAI:master Nov 29, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants