Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add generic New Classes scenario manager #39

Closed
lrzpellegrini opened this issue May 12, 2020 · 1 comment · Fixed by #64
Closed

Add generic New Classes scenario manager #39

lrzpellegrini opened this issue May 12, 2020 · 1 comment · Fixed by #64
Assignees
Labels
Feature - High Priority New feature or request, high priority

Comments

@lrzpellegrini
Copy link
Collaborator

Already implemented in my private codebase, working on porting it to Avalanche.

This class will allow the user to create a NC (New Classes) scenario given a couple of generic train and test Datasets.

The user will be able to create a manager instance that will be an iterable. This iterable will output the incremental "task"s or "batch"es (terminology to be defined) and will also allow the user to execute certain task/batch complex management operations.

This is very similar to the current loader being implemented in the Avalanche codebase, but will allow the user to plug in his/her own dataset. Also, due to being extremely generic, this will speed-up the integration of new datasets.

The code I've already implemented in my private codebase works fine, but is complex. I'm working on slimming it down a bit. Here is a list of already implemented features, please feel free to comment if you feel we need even more features!

Features (already implemented):

  • Get current/cumulative/task-specific train/test datasets
  • Variable number of tasks (or "incremental batches" for task-free scenarios)
  • Allow the user to customize the number of classes in each task
  • Class shuffling given a seed
  • Ability to define a fixed class order (for results reproducibility)
  • Remapping class original IDs to range(0, n_classes) (very useful when creating confusion matrices and for algorithms based on dynamic head expansion which require class idxs in ascending order)

Side features (already implemented):

  • Wrapper Dataset class that makes any Dataset sliceable and funny-indexable
  • Wrapper Dataset class that makes any Dataset transformable (like the ones in torchvision)

To be defined (even in future development phases):

  • Terminology: the manager will output "task"s. That is, training/test sets made only of patterns of certain classes. The question is: is the "task" terminology "ok"? Or is too much related to task-oriented scenarios? Consider that, apart from the terminology considerations, the users will be able to use this manager both in multi-task and task-free (single incremental task) scenarios...

Feel free to comment.

Keep up with the excellent work you've been doing!

@lrzpellegrini lrzpellegrini self-assigned this May 12, 2020
@vlomonaco
Copy link
Member

Thanks @lrzpellegrini for this, I think we can start by adding this calss as an additional Avalanche feature, than we will see if we can use it as a parent class for all the data loaders.

As for the nomenclature, i think the word batch is more general...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature - High Priority New feature or request, high priority
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants