Skip to content
Switch branches/tags
Go to file

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time

Skip Connections Matter

This repository contains the code for Skip Connections Matter: On the Transferability of Adversarial Examples Generated with ResNets (ICLR 2020 Spotlight).


  • 11/18/2020 - We released more codes and the dataset used in our paper(a subset with 5000 images from ImageNet), to help reproduce the reported results in our paper.
  • 02/20/2020 - arXiv posted and repository released.


We propose the Skip Gradient Method (SGM) to generate adversarial examples using gradients more from the skip connections rather than the residual modules. In particular, SGM utilizes a decay factor (gamma) to reduce gradients from the residual modules,


This code is implemented in PyTorch, and we have tested the code under the following environment settings:

  • python = 3.7.6
  • torch = 1.7.0
  • torchvision = 0.8.1
  • advertorch = 0.2.2
  • pretrainedmodels = 0.7.4

Run the code

  1. Download the dataset from Google Drive or Baidu Drive (pw:55rk), and extract images to the path ./SubImageNet224/
  2. Generate adversarial examples and save them into path ./adv_images/. For ResNet-152 as the source model,
    python --gamma 0.2 --output_dir adv_images --arch densenet201 --batch-size 40
    For DenseNet-201 as the source model,
    python --gamma 0.5 --output_dir adv_images --arch resnet152 --batch-size 40
  3. Evaluate the transerability of generated adversarial examples in ./adv_images/. For VGG19 with batch norm as the target model
    python --input_dir adv_images --arch vgg19_bn


  • Visualization

  • Reproduced results

We run this code, and the attack success (1 - acc) against VGG19 is close to the repored in our paper:

Source \ Method PGD MI SGM
ResNet-152 45.80% 66.70% 81.04%
DenseNet-201 57.82% 75.38% 82.58%


For easier reproduction, we provide more detailed information here.

register backward hook for SGM

In fact, we manipulate gradients flowing through ReLU in utils_sgm, since there is no ReLU in skip-connections:

  • For ResNet, there are "downsampling" modules in which skip-connections are replaced by a conv layer. We do not manipulate gradients of "downsampling" module;

  • For DenseNet, we manipulate gradients in all dense block.

Pretrained models

All pretrained models in our paper can be found online:

Citing this work

    title={Skip connections matter: On the transferability of adversarial examples generated with resnets},
    author={Wu, Dongxian and Wang, Yisen and Xia, Shu-Tao and Bailey, James and Ma, Xingjun},


[1] Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boosting adversarial attacks with momentum. In CVPR, 2018.

[2] Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, Patrick McDaniel. Ensemble Adversarial Training: Attacks and Defenses. In ICLR, 2018.


Codes for ICLR 2020 paper "Skip Connections Matter: On the Transferability of Adversarial Examples Generated with ResNets"





No releases published


No packages published