Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rework baselines #25

Merged
merged 23 commits into from
Jun 2, 2023
Merged

Rework baselines #25

merged 23 commits into from
Jun 2, 2023

Conversation

alafage
Copy link
Contributor

@alafage alafage commented May 29, 2023

Current baseline implementations carry lots of repetitions. To simplify them this pull request proposes the following changes:

  1. Classification Single/Ensemble routines are no more abstract. They can be used directly with models, losses, etc.
  2. Unique baseline for each backbone (ResNet, WideResNet, etc.) where the version (ex: vanilla, packed, etc.) is passed as an argument.

These changes are expected to:

  1. reduce considerably the number of files for the baselines
  2. ease the use of routines to train custom models
  • Routines are not more abstract
  • ResNet baseline
  • WideResNet baseline
  • ResNet experiments
  • WideResNet experiments
  • Custom optimizer management for BatchEnsemble
  • Simplify the arguments (of packed, etc) but also of the routine (metrics)
  • Simplify the argument consistency tests
  • Tests
  • Documentation

@alafage alafage changed the base branch from main to dev May 29, 2023 16:18
@o-laurent
Copy link
Contributor

o-laurent commented May 29, 2023

You also moved loading weights inside the model! One step further to solving #11

@o-laurent o-laurent self-requested a review May 29, 2023 18:28
@o-laurent o-laurent added documentation Improvements or additions to documentation enhancement New feature or request labels May 30, 2023
if ds_name == "cifar10" or ds_name == "cifar100":
procedure = optim_cifar10_wideresnet

if model_name == "batch_ensemble":
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it should be:

if model_name == "batched":
    ...

Copy link
Contributor

@o-laurent o-laurent May 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right. However, is batched the correct name? Calling BatchEnsemble "batched" may amount to some form of appropriation. (says the guy that called BatchEnsemble "BatchEnsembles" twice in the commits 😆)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I fixed the optimizer, but we may need to look at the names we gave to the different methods.

@o-laurent o-laurent mentioned this pull request May 30, 2023
8 tasks
@alafage alafage marked this pull request as ready for review June 2, 2023 07:52
Copy link
Contributor

@o-laurent o-laurent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Starting to look nice!

experiments/classification/cifar100/resnet.py Outdated Show resolved Hide resolved
experiments/classification/readme.md Show resolved Hide resolved
experiments/readme.md Outdated Show resolved Hide resolved
torch_uncertainty/layers/packed_layers.py Show resolved Hide resolved
@alafage alafage merged commit 94461c1 into dev Jun 2, 2023
1 check passed
@alafage alafage deleted the rework-baselines branch June 2, 2023 11:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants