Skip to content

MadryLab/BREEDS-Benchmarks

master
Switch branches/tags
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 

Creating BREEDS Sub-population shift Benchmarks

This repository contains the code and data for our paper:

BREEDS: Benchmarks for Subpopulation Shift
Shibani Santurkar*, Dimitris Tsipras*, Aleksander Madry
Paper: https://arxiv.org/abs/2008.04859

    @InProceedings{santurkar2020breeds,
        title={BREEDS: Benchmarks for Subpopulation Shift},
        author={Shibani Santurkar and Dimitris Tsipras and Aleksander Madry},
        year={2020},
        booktitle={ArXiv preprint arXiv:2008.04859}
    }

Getting started

Our code relies on the MadryLab public robustness library, which will be automatically installed when you follow the instructions below.

  1. Clone our repo: git clone https://github.com/MadryLab/BREEDS-Benchmarks.git

  2. Install dependencies:

    conda create -n breeds-benchmarks python=3.7 pip
    conda activate breeds-benchmarks
    pip install -r requirements.txt
    conda install pygraphviz
    
  3. Download the ImageNet dataset.

That's it! Now you can create new BREEDS datasets, or look closer at the ones we use in our paper using this notebook. You can also find a detailed walkthrough of this code in this documentation. In order to train models using these datasets, you can use simply use existing code from the robustness library.

Maintainers

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published