Skip to content

Conversation

@lrzpellegrini
Copy link
Collaborator

This PR adds the support for custom streams in benchmark creation helpers (the functions commonly used to create a benchmark instance).

This adds the following changes to helpers:

  • The new helpers support the definition of custom streams. Because of that the API has been slightly modified.
  • The new helpers have been implemented in 2 separate scripts. Old helpers are perfectly functioning but they will now display a deprecation warning at runtime.
  • Names of new helpers follow the nomenclature found in the Avalanche paper (https://arxiv.org/abs/2104.00405)

Classic benchmarks, unit tests and other parts of the code of Avalanche no longer use the deprecated helpers.

This is a draft PR: documentation and notebooks have not been adapted yet.

Also, consider that the old nomenclature is still in place, mainly in variable names. Switching to the new one will take some more time (will probably create a separate issue for that).

@coveralls
Copy link

Pull Request Test Coverage Report for Build 764295843

  • 313 of 418 (74.88%) changed or added relevant lines in 28 files are covered.
  • 99 unchanged lines in 3 files lost coverage.
  • Overall coverage decreased (-1.4%) to 74.691%

Changes Missing Coverage Covered Lines Changed/Added Lines %
avalanche/benchmarks/classic/cmnist.py 4 5 80.0%
avalanche/benchmarks/classic/core50.py 1 2 50.0%
avalanche/benchmarks/classic/ctiny_imagenet.py 2 3 66.67%
avalanche/benchmarks/classic/openloris.py 1 2 50.0%
avalanche/benchmarks/classic/stream51.py 1 2 50.0%
avalanche/benchmarks/classic/ccifar10.py 1 3 33.33%
avalanche/benchmarks/classic/ccub200.py 1 3 33.33%
avalanche/benchmarks/classic/cimagenet.py 1 3 33.33%
avalanche/benchmarks/classic/ccifar100.py 1 4 25.0%
tests/unit_tests_utils.py 3 6 50.0%
Files with Coverage Reduction New Missed Lines %
avalanche/benchmarks/utils/avalanche_dataset.py 2 85.28%
avalanche/benchmarks/scenarios/generic_scenario_creation.py 42 23.94%
avalanche/benchmarks/generators/scenario_generators.py 55 24.72%
Totals Coverage Status
Change from base Build 758281065: -1.4%
Covered Lines: 6717
Relevant Lines: 8993

💛 - Coveralls

@lrzpellegrini lrzpellegrini marked this pull request as ready for review April 20, 2021 15:46
@lrzpellegrini
Copy link
Collaborator Author

I adapted the notebook, fixed few bugs and added some unit tests. I think it's ready.

@AntonioCarta
Copy link
Collaborator

Looks good to me. If it's ok for the others, you can merge when you are ready.

@lrzpellegrini lrzpellegrini merged commit 4ee42e6 into ContinualAI:master Apr 21, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants