Vibcreg

[Froskekongen]

Implementation of VIbCReg (https://arxiv.org/abs/2109.00783), which was originally developed for times series modeling.

The performance without tuning parameters and using the same parameters as VICReg gets a performance of 91.18 acc@1 online, and 99.74 acc@5 online.

[vturrisi]

Hey @Froskekongen thanks a lot for contributing. I'll take a look at the paper first to get the general idea.
It might take longer than normal because of CVPR, but I'll try to give some feedback as soon as I can.

[DonkeyShot21]

Hi @Froskekongen! We appreciate your initiative.

Did you happen to run CIFAR100 as well?

[Froskekongen]

@DonkeyShot21: I will run CIFAR-100 overnight.

EDIT: I will run it overnight if I manage to download the dataset. There appears to be a 500 error from the site hosting the dataset right now.

[vturrisi]

I found some small things that would be good to be fixed before merging.
Also, can you add the loss to solo/losses/__init__.py?

TODO:

• implement tests
• add method to the docs
• add tests
• add results/checkpoints to README
• black files (the bot seems to have failed for some reason)

[vturrisi]

I'm getting this error. It's probably related to the clone() that I said we could remove. Sorry about that.

E RuntimeError: unsupported operation: more than one element of the written-to tensor refers to a single memory location. Please clone() the tensor before performing the operation.

[Froskekongen]

I reintroduced the .clone().

[vturrisi]

@Froskekongen I cleaned a bit, added some placeholders for results in the readme and fixed part of the tests. We are still getting an error which I assume is due to the clone() that I asked you to remove. We are also missing documentations (me or @DonkeyShot21 can add later) and the results/checkpoints for CIFAR10/100 and Imagenet-100.

[codecov]

Codecov Report

Merging #182 (885bf8c) into main (a829643) will decrease coverage by 0.10%.
The diff coverage is 94.11%.

Flag	Coverage Δ		*Carryforward flag
cpu	84.56% <94.64%> (+0.59%)	⬆️
dali	42.24% <80.00%> (ø)		Carriedforward from dae451c

*This pull request uses carry forward flags. Click here to find out more.

Impacted Files	Coverage Δ
solo/utils/whitening.py	88.14% <90.47%> (-11.86%)	⬇️
solo/losses/__init__.py	100.00% <100.00%> (ø)
solo/losses/vibcreg.py	100.00% <100.00%> (ø)
solo/methods/__init__.py	93.10% <100.00%> (-6.90%)	⬇️
solo/methods/vibcreg.py	100.00% <100.00%> (ø)

[vturrisi]

I'll take care of the documents. @Froskekongen are you running/planning to run the experiments?

[Froskekongen]

@vturrisi: I ran the experiments for cifar10 and cifar100
cifar10: val_acc1 - 91.18, val_acc5 - 99.74
cifar100: val_acc1 - 67.37, val_acc5 - 90.07

[vturrisi]

Ok! I'll create some folders so that you can upload the checkpoints, k? About imagenet100, are you running or planning to run?

[Froskekongen]

Yes, I can upload the trained models to some folders. I didn't plan to run imagenet100 - at least not right now.

[vturrisi]

@Froskekongen just to confirm we are missing the checkpoints and Imagenet-100 results.
You can add the cifar10 checkpoint here: https://drive.google.com/drive/folders/19u3p1maX3xqwoCHNrqSDb98J5fRvd_6v?usp=sharing and cifar100 here: https://drive.google.com/drive/folders/19u3p1maX3xqwoCHNrqSDb98J5fRvd_6v?usp=sharing, both the args.json file and the checkpoint itself. When I'm able to run imagenet-100, I'll upload everything and we merge the PR.

EDIT: just realised that I put the same link twice. I already moved the files to the correct folders.

[Froskekongen]

I have added the args and checkpoints to the drive folder.

[vturrisi]

Hey @Froskekongen. I'll run on imagenet-100. Can I use the same parameters as in VICReg?

[Froskekongen]

@vturrisi: The only thing that needs to be changed are the loss weights, which should be:

    --sim_loss_weight 25.0 \
    --var_loss_weight 25.0 \
    --cov_loss_weight 200.0 \

[vturrisi]

@Froskekongen just started with imagenet-100. When this is done and I run offline linear eval, I'll merge the PR (that should be tomorrow or Monday).

[vturrisi]

Merging it now. Thanks again @Froskekongen for contributing.