Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement RSC (lightly tested) / Cleanup hyperparameter registry #12

Merged
merged 8 commits into from
Sep 15, 2020
Merged

Conversation

SirRob1997
Copy link
Contributor

@SirRob1997 SirRob1997 commented Sep 8, 2020

This PR implements the following features:

  • Representation Self-Challenging (RSC) from: https://arxiv.org/abs/2007.02454
  • Cleanup algorithm-specific hyperparameters in hyperparameter registry so they don't flood the console by default
  • Adapted MNIST_CNN forward pass to use AdaptiveAvgpool2d instead of mean, this allows for more flexible usage in algorithms and feels more clean

You might also want to delete the MLP parts in networks.py and hparams_registry.py as well since I think MLP was only added for Debug datasets that don't get used anymore?

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 8, 2020
@SirRob1997 SirRob1997 changed the title Cleanup algorithm specific hyperparameters in hyperparameter registry Implement RSC / Cleanup algorithm specific hyperparameters in hyperparameter registry Sep 12, 2020
@SirRob1997 SirRob1997 changed the title Implement RSC / Cleanup algorithm specific hyperparameters in hyperparameter registry Implement RSC / Cleanup hyperparameter registry Sep 12, 2020
@SirRob1997 SirRob1997 changed the title Implement RSC / Cleanup hyperparameter registry Implement RSC (lightly tested) / Cleanup hyperparameter registry Sep 12, 2020
domainbed/algorithms.py Outdated Show resolved Hide resolved
domainbed/networks.py Outdated Show resolved Hide resolved
domainbed/networks.py Outdated Show resolved Hide resolved
domainbed/networks.py Outdated Show resolved Hide resolved
@lopezpaz
Copy link
Contributor

Note: MLP is used in DANN variants.

@SirRob1997
Copy link
Contributor Author

Then the MLP parameters should only show in the console when DANN is used as an algorithm

@lopezpaz lopezpaz merged commit ba2d9d6 into facebookresearch:master Sep 15, 2020
@lopezpaz
Copy link
Contributor

Merged, although we will have to find a solution to guarantee that the same random parameters are sampled in hparams_registry given the same random seed, for all future commits. Right now, adding algorithms or conditional logic in hparams_registry changes the random hyperparameters sampled for a given algorithm, dataset, and seed.

cc @igul222

@lopezpaz
Copy link
Contributor

@SirRob1997: Preliminar runs show a very poor performance (39%) at OfficeHome. The paper reports 63%. Could you investigate how to get a higher number in this dataset with default hyper-parameters?

daysm pushed a commit to daysm/DomainBed that referenced this pull request Nov 19, 2020
Implement RSC (lightly tested) / Cleanup hyperparameter registry
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants