ZOFC

This is implmentation of our WACV2026 paper: [More Than Memory Savings: Zeroth-Order Optimization Mitigates Forgetting in Continual Learning].

how to use

Dependencies

1. torch 2.0.1
2. torchvision 0.15.2
3. timm 0.6.12
4. tqdm
5. numpy 1.23.5
6. scipy
7. easydict

Also, see requirements.txt

Run experiment

1. Edit the [MODEL NAME].json file for global settings and hyperparameters.

2. Run:

   python main.py --config=./exps/[MODEL NAME].json

For ZOFC reproductions, you can first run

1. python main.py --config=./exps/newlae_hybrid_zafc_cifar_5.json
2. python main.py --config=./exps/newlae_hybrid_zafc_cifar_10.json
3. python main.py --config=./exps/newlae_hybrid_zafc_inr_5.json
4. python main.py --config=./exps/newlae_hybrid_zafc_inr_10.json

Datasets

We use Cifar100, ImageNet-R, DomainNet.

• CIFAR100: will be automatically downloaded by the code.
• ImageNet-R: Google Drive: link or Onedrive: link
• DomainNet: download from link, place it into data/ folder

These subsets are sampled from the original datasets. Please note that I do not have the right to distribute these datasets. If the distribution violates the license, I shall provide the filenames instead.

When training not on CIFAR100, you should specify the folder of your dataset in utils/data.py.

    def download_data(self):
        assert 0,"You should specify the folder of your dataset"
        train_dir = '[DATA-PATH]/train/'
        test_dir = '[DATA-PATH]/val/'

Acknowledgement

We would like to thank the following repos providing helpful components/functions in our work:

• [PILOT].
• [InfLoRA].