Skip to content

microsoft/robust-models-transfer

master
Switch branches/tags
Code

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
src
 
 
 
 
 
 
 
 
 
 

Transfer Learning using Adversarially Robust ImageNet Models

This repository contains the code and models necessary to replicate the results of our paper:

Do Adversarially Robust ImageNet Models Transfer Better?
Hadi Salman*, Andrew Ilyas*, Logan Engstrom, Ashish Kapoor, Aleksander Madry
Paper: https://arxiv.org/abs/2007.08489
Blog post: https://www.microsoft.com/en-us/research/blog/adversarial-robustness-as-a-prior-for-better-transfer-learning/

    @InProceedings{salman2020adversarially,
        title={Do Adversarially Robust ImageNet Models Transfer Better?},
        author={Hadi Salman and Andrew Ilyas and Logan Engstrom and Ashish Kapoor and Aleksander Madry},
        year={2020},
        booktitle={ArXiv preprint arXiv:2007.08489}
    }

Getting started

Our code relies on the MadryLab public robustness library, which will be automatically installed when you follow the instructions below.

  1. Clone our repo: git clone https://github.com/microsoft/robust-models-transfer.git

  2. Install dependencies:

    conda create -n robust-transfer python=3.7
    conda activate robust-transfer
    pip install -r requirements.txt
    

Running transfer learning experiments

The entry point of our code is main.py (see the file for a full description of arguments).

1- Download one of the pretrained robust ImageNet models, say an L2-robust ResNet-18 with ε = 3. For a full list of models, see the section below!

mkdir pretrained-models & 
wget -O pretrained-models/resnet-18-l2-eps3.ckpt "https://robustnessws4285631339.blob.core.windows.net/public-models/robust_imagenet/resnet18_l2_eps3.ckpt?sv=2020-08-04&ss=bfqt&srt=sco&sp=rwdlacupitfx&se=2051-10-06T07:09:59Z&st=2021-10-05T23:09:59Z&spr=https,http&sig=U69sEOSMlliobiw8OgiZpLTaYyOA5yt5pHHH5%2FKUYgI%3D"

2- Run the following script (best parameters for each dataset and architecture can be found here)

python src/main.py --arch resnet18 \
  --dataset cifar10 \
  --data /tmp \
  --out-dir outdir \
  --exp-name cifar10-transfer-demo \
  --epochs 150 \
  --lr 0.01 \
  --step-lr 30 \
  --batch-size 64 \
  --weight-decay 5e-4 \
  --adv-train 0 \
  --model-path pretrained-models/resnet-18-l2-eps3.ckpt
  --freeze-level -1

--freeze-level -> -1: full-network transfer | 4: fixed-feature transfer

3- That's it!

Datasets that we use (see our paper for citations)

To use any of these datasets in the code:

  1. Download (click or use wget) and extract the desired dataset somewhere, e.g.
    tar -xvf pets.tar
    mkdir /tmp/datasets
    mv pets /tmp/datasets
    
  2. Add the dataset name and path as arguments , e.g.
    python src/main.py --arch resnet18  ... --dataset pets --data /tmp/datasets/pets
    

Architectures

You can choose an architecture to use by simply passing it as arguments to the code e.g.

python src/main.py --arch resnet50 ...

The set of possible architectures is:

archs = [resnet18, 
         resnet50, 
         wide_resnet50_2, 
         wide_resnet50_4, 
         densenet,
         mnasnet,
         mobilenet,
         resnext50_32x4d,
         shufflenet,
         vgg16_bn
         ]

Download our robust ImageNet models

If you find our pretrained models useful, please consider citing our work.

Standard Accuracy of L2-Robust ImageNet Models

Model ε=0 ε=0.01 ε=0.03 ε=0.05 ε=0.1 ε=0.25 ε=0.5 ε=1.0 ε=3.0 ε=5.0
ResNet-18 69.79 69.90 69.24 69.15 68.77 67.43 65.49 62.32 53.12 45.59
ResNet-50 75.80 75.68 75.76 75.59 74.78 74.14 73.16 70.43 62.83 56.13
Wide-ResNet-50-2 76.97 77.25 77.26 77.17 76.74 76.21 75.11 73.41 66.90 60.94
Wide-ResNet-50-4 77.91 78.02 77.87 77.77 77.64 77.10 76.52 75.51 69.67 65.20
Model ε=0 ε=3
DenseNet 77.37 66.98
MNASNET 60.97 41.83
MobileNet-v2 65.26 50.40
ResNeXt50_32x4d 77.38 66.25
ShuffleNet 64.25 43.32
VGG16_bn 73.66 57.19

Standard Accuracy of Linf-Robust ImageNet Models

Model ε=0.5/255 ε=1/255 ε=2/255 ε=4/255 ε=8/255
ResNet-18 66.13 63.46 59.63 52.49 42.11
ResNet-50 73.73 72.05 69.10 63.86 54.53
Wide-ResNet-50-2 75.82 74.65 72.35 68.41 60.82

Maintainers

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.

About

Official repository for our NeurIPS 2020 *oral* "Do Adversarially Robust ImageNet Models Transfer Better?"

Resources

License

Code of conduct

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages