Revisiting Transferable Adversarial Images: Systemization, Evaluation, and New Insights. Zhengyu Zhao*, Hanwei Zhang*, Renjue Li*, Ronan Sicre, Laurent Amsaleg, Michael Backes, Qi Li, Qian Wang, Chao Shen.
We identify two main problems in common evaluation practices:
(1) for attack transferability, lack of systematic, one-to-one attack comparisons and fair hyperparameter settings;
(2) for attack stealthiness, simply no evaluations.
We address these problems by
(1) introducing a complete attack categorization and conducting systematic and fair intra-category analyses on transferability;
(2) considering diverse imperceptibility metrics and finer-grained stealthiness characteristics from the perspective of attack traceback.
We draw new insights, e.g.,
(1) under a fair attack hyperparameter setting, one early attack method, DI, actually outperforms all the follow-up methods;
(2) popular diffusion-based defenses give a false sense of security since it is indeed largely bypassed by (black-box) transferable attacks;
(3) even when all attacks are bounded by the same Lp norm, they lead to dramatically different stealthiness performance, which negatively correlates with their transferability performance.
We provide the first large-scale evaluation of transferable adversarial examples on ImageNet, involving 23 representative attacks against 9 representative defenses.
We reveal that existing problematic evaluations have indeed caused misleading conclusions and missing points, and as a result, hindered the assessment of the actual progress in this field.
Gradient Stabilization Attacks [Code for 3 representative attacks]
- Boosting Adversarial Attacks with Momentum (CVPR 2018)
- Nesterov Accelerated Gradient and Scale Invariance for Adversarial Attacks (ICLR 2020)
- Boosting Adversarial Transferability through Enhanced Momentum (BMVC 2021)
- Improving Adversarial Transferability with Spatial Momentum (arXiv 2022)
- Making Adversarial Examples More Transferable and Indistinguishable (AAAI2022)
- Boosting Adversarial Transferability by Achieving Flat Local Maxima (NeurIPS 2023)
- Transferable Adversarial Attack for Both Vision Transformers and Convolutional Networks via Momentum Integrated Gradients (ICCV 2023)
- Boosting Adversarial Transferability via Gradient Relevance Attack (ICCV 2023)
- Enhancing Transferable Adversarial Attacks on Vision Transformers through Gradient Normalization Scaling and High-Frequency Adaptation (ICLR 2024)
Input Augmentation Attacks [Code for 5 representative attacks]
- Improving Transferability of Adversarial Examples with Input Diversity (CVPR 2019)
- Evading Defenses to Transferable Adversarial Examples by Translation-Invariant Attacks (CVPR 2019)
- Nesterov Accelerated Gradient and Scale Invariance for Adversarial Attacks (ICLR 2020)
- Patch-wise Attack for Fooling Deep Neural Network (ECCV 2020)
- Improving the Transferability of Adversarial Examples with Resized-Diverse-Inputs, Diversity-Ensemble and Region Fitting (ECCV 2020)
- Regional Homogeneity: Towards Learning Transferable Universal Adversarial Perturbations Against Defenses (ECCV 2020)
- Enhancing the Transferability of Adversarial Attacks through Variance Tuning (CVPR 2021)
- Admix: Enhancing the Transferability of Adversarial Attacks (ICCV 2021)
- Improving the Transferability of Targeted Adversarial Examples through Object-Based Diverse Input (CVPR 2022)
- Frequency Domain Model Augmentation for Adversarial Attack (ECCV 2022)
- Adaptive Image Transformations for Transfer-based Adversarial Attack (ECCV 2022)
- Boosting the Transferability of Adversarial Attacks with Reverse Adversarial Perturbation (NeurIPS 2022)
- Enhancing the Self-Universality for Transferable Targeted Attacks (CVPR 2023)
- Improving the Transferability of Adversarial Samples by Path-Augmented Method (CVPR 2023)
- The Ultimate Combo: Boosting Adversarial Example Transferability by Composing Data Augmentations (arXiv 2023)
- Structure Invariant Transformation for better Adversarial Transferability (ICCV 2023)
- Boosting Adversarial Transferability across Model Genus by Deformation-Constrained Warping (AAAI 2024)
- Boosting Adversarial Transferability by Block Shuffle and Rotation (CVPR 2024)
- Learning to Transform Dynamically for Better Adversarial Transferability (CVPR 2024)
- Boosting the Transferability of Adversarial Examples via Local Mixup and Adaptive Step Size (arXiv 2024)
- Typography Leads Semantic Diversifying: Amplifying Adversarial Transferability across Multimodal Large Language Models (arXiv 2024)
- Strong Transferable Adversarial Attacks via Ensembled Asymptotically Normal Distribution Learning (CVPR 2024)
- Learning to Transform Dynamically for Better Adversarial Transferability (CVPR2024)
Feature Disruption Attacks [Code for 5 representative attacks]
- Transferable Adversarial Perturbations (ECCV 2018)
- Task-generalizable Adversarial Attack based on Perceptual Metric (arXiv 2018)
- Feature Space Perturbations Yield More Transferable Adversarial Examples (CVPR 2019)
- FDA: Feature Disruptive Attack (ICCV 2019)
- Enhancing Adversarial Example Transferability with an Intermediate Level Attack (ICCV 2019)
- Transferable Perturbations of Deep Feature Distributions (ICLR 2020)
- Boosting the Transferability of Adversarial Samples via Attention (CVPR 2020)
- Towards Transferable Targeted Attack (CVPR 2020)
- Yet Another Intermediate-Level Attack (ECCV 2020)
- Perturbing Across the Feature Hierarchy to Improve Standard and Strict Blackbox Attack Transferability (NeurIPS 2020)
- Feature Importance-aware Transferable Adversarial Attacks (ICCV 2021)
- Improving Adversarial Transferability via Neuron Attribution-Based Attacks (CVPR 2022)
- An Intermediate-level Attack Framework on The Basis of Linear Regression (TPAMI 2022)
- Introducing Competition to Boost the Transferability of Targeted Adversarial Examples through Clean Feature Mixup (CVPR 2023)
- Diversifying the High-level Features for better Adversarial Transferability (BMVC 2023)
- Improving Adversarial Transferability via Intermediate-level Perturbation Decay (NeurIPS 2023)
- Boosting Adversarial Transferability via Fusing Logits of Top-1 Decomposed Feature (arXiv 2023)
Surrogate Refinement Attacks [Code for 5 representative attacks]
- Learning Transferable Adversarial Examples via Ghost Networks (AAAI 2020)
- Skip Connections Matter: On the Transferability of Adversarial Examples Generated with ResNets (ICLR 2020)
- Backpropagating Linearly Improves Transferability of Adversarial Examples (NeurIPS 2020)
- Backpropagating Smoothly Improves Transferability of Adversarial Examples (CVPRw 2021)
- A Little Robustness Goes a Long Way: Leveraging Robust Features for Targeted Transfer Attacks (NeurIPS 2021)
- Early Stop and Adversarial Training Yield Better Surrogate Model: Very Non-Robust Features Harm Adversarial Transferability (OpenReview 2021)
- Stochastic Variance Reduced Ensemble Adversarial Attack for Boosting the Adversarial Transferability (CVPR 2022)
- Training Meta-Surrogate Model for Transferable Adversarial Attack (arXiv 2021)
- On Improving Adversarial Transferability of Vision Transformers (ICLR2022)
- Rethinking Adversarial Transferability from a Data Distribution Perspective (ICLR 2022)
- Boosting the Adversarial Transferability of Surrogate Model with Dark Knowledge (arXiv 2022)
- Towards Transferable Adversarial Attacks on Vision Transformers (AAAI 2022)
- Transferable Adversarial Attacks on Vision Transformers with Token Gradient Regularization (CVPR 2023)
- Minimizing Maximum Model Discrepancy for Transferable Black-box Targeted Attacks (CVPR 2023)
- Towards Transferable Targeted Adversarial Examples (CVPR 2023)
- StyLess: Boosting the Transferability of Adversarial Examples (CVPR 2023)
- How to choose your best allies for a transferable attack? (ICCV 2023)
- An Adaptive Model Ensemble Adversarial Attack for Boosting Adversarial Transferability (ICCV 2023)
- Backpropagation Path Search On Adversarial Transferability (ICCV 2023)
- Going Further: Flatness at the Rescue of Early Stopping for Adversarial Example Transferability (arXiv 2023)
- Making Substitute Models More Bayesian Can Enhance Transferability of Adversarial Examples (ICLR 2023)
- Blurred-Dilated Method for Adversarial Attacks (NeurIPS 2023)
- Rethinking the Backward Propagation for Adversarial Transferability (NeurIPS 2023)
- Rethinking Model Ensemble in Transfer-based Adversarial Attacks (ICLR 2024)
- Why Does Little Robustness Help? Understanding and Improving Adversarial Transferability from Surrogate Training (S&P 2024)
- Enhance Stealthiness and Transferability of Adversarial Attacks with Class Activation Mapping Ensemble Attack (NDSS 2024)
- Improving Transferable Targeted Adversarial Attacks with Model Self-Enhancement (CVPR 2024)
- AGS: Affordable and Generalizable Substitute Training for Transferable Adversarial Attack (AAAI 2024)
- Ensemble Diversity Facilitates Adversarial Transferability (CVPR 2024)
- Generative Adversarial Perturbations (CVPR 2018)
- Cross-Domain Transferability of Adversarial Perturbations (NeurIPS 2019)
- On Generating Transferable Targeted Perturbations (ICCV 2021)
- Learning Transferable Adversarial Perturbations (NeurIPS 2021)
- Beyond ImageNet Attack: Towards Crafting Adversarial Examples for Black-box Domains (ICLR 2022)
- Boosting Transferability of Targeted Adversarial Examples via Hierarchical Generative Networks (ECCV 2022)
- Dynamic Generative Targeted Attacks With Pattern Injection (CVPR 2023)
- Towards Transferable Targeted Adversarial Examples (CVPR 2023)
- Perturbation Towards Easy Samples Improves Targeted Adversarial Transferability (NeurIPS 2023)
- Delving into Transferable Adversarial Examples and Black-box Attacks (ICLR 2017)
- The Space of Transferable Adversarial Examples (arXiv 2017)
- Why Do Adversarial Attacks Transfer? Explaining Transferability of Evasion and Poisoning Attacks (USENIX Security 2019)
- Selection of Source Images Heavily Influences the Effectiveness of Adversarial Attacks (BMVC 2021)
- On Success and Simplicity: A Second Look at Transferable Targeted Attacks (NeurIPS 2021)
- Evaluating Adversarial Attacks on ImageNet: A Reality Check on Misclassification Classes (NeurIPSw 2021)
- A Unified Approach to Interpreting and Boosting Adversarial Transferability (ICLR 2021)
- Transfer Attacks Revisited: A Large-Scale Empirical Study in Real Computer Vision Settings (IEEE S&P 2022)
- Towards Evaluating Transfer-based Attacks Systematically, Practically, and Fairly (NeurIPS 2023)
- Beyond Boundaries: A Comprehensive Survey of Transferable Attacks on AI Systems (arXiv 2023)
- A Survey on Transferability of Adversarial Examples across Deep Neural Networks (arXiv 2023)
- Bag of Tricks to Boost Adversarial Transferability (arXiv 2024)