This is the official starting repository for the Continual Learning Challenge held in the 3rd CLVision Workshop @ CVPR 2022.
Please refer to the challenge website for more details!
The Slack channel #3rd_clvision_challenge is now open in the ContinualAI Slack: join here.
We are gathering feedback about the devkit. Join the Slack channel or open an issue directly on this repo!
The devkit is based on the Avalanche library. We warmly recommend looking at the documentation (especially the "Zero To Hero tutorials") if this is your first time using it!
For the demo track, Avalanche is added as a Git submodule of this repository. The submodule points to a specific commit of a branch we specifically created for the challenge.
The recommended setup steps are as follows:
git clone --recurse-submodules https://github.com/ContinualAI/clvision-challenge-2022.git
cd clvision-challenge-2022
conda env create -f environment.yml-
Setup your IDE so that the avalanche submodule is included in the PYTHONPATH. Note: you have to include the top-level folder, not
avalanche/avalanche!- For Jetbrains IDEs (PyCharm), this can be done from the Project pane (usually on the right) by right-clicking on the "avalanche" folder -> "Mark Directory as" -> "Sources Root".
- For VS Code, follow the official documentation.
-
Download and extract the dataset: in order to download the dataset, we ask all participants to accept the dataset terms and provide their email addresses through this form. You will immediately receive the download instructions at the provided address. We recommend extracting the dataset in the default folder
$HOME/3rd_clvision_challenge/demo_dataset/. The final directory structure should be like this:
$HOME/3rd_clvision_challenge/demo_dataset/
├── ego_objects_demo_test.json
├── ego_objects_demo_train.json
├── images
│ ├── 07A28C4666133270E9D65BAB3BCBB094_0.png
│ ├── 07A28C4666133270E9D65BAB3BCBB094_100.png
│ ├── 07A28C4666133270E9D65BAB3BCBB094_101.png
│ ├── ...
The aforementioned steps should be OS-agnostic. However, we recommend setting up your dev environment using a mainstream Linux distro.
The starting template for the object classification track is based on the tried and tested strategies offered by Avalanche.
In particular, the starting template can be found in starting_template_track1.py. The
default template offers a working train/eval loop that uses the
Naive strategy.
The Naive strategy is a plain fine-tuning loop that, given the optimizer, number of epochs, the minibatch size, and the loss function, will just run a very forgetting-prone training loop. This should be taken as the lower bound for a solution. This means that the basic loop already there, ready to be customized. There are two main ways to implement your solution:
- Override parts of the base class
SupervisedTemplatein order to customize the epoch loop, the backward and forward operations, etcetera. - Implement your solution as a plugin (many mainstream techniques are implemented in Avalanche as plugins, see the documentation)
We suggest you to study the From Zero To Hero tutorials to learn about Avalanche.
The starting point for the detection tracks is starting_template_track2.py.
That entry point uses the (DetectionTemplate) template, which is the template you should customize
to implement your CL strategy.
That training template is based on Avalanche training templates. The training loop is an almost exact implementation of the one shown in the official TorchVision Object Detection Finetuning Tutorial, especially the train_one_epoch method.
A schematic visualization of the training loop, its events, and an example of a plugin implementing EWC is shown below:
- During the demo phase, the devkit will be updated quite often. We recommend checking if there are new updates frequently.
- The default logger will just print the progress to stdout (and it is quite verbose). Consider using dashboard loggers, such as Tensorboard or Weights & Biases. See the tutorial on loggers here. You can use more than one logger at the same time!