This repository includes open-source codes, detailed experimental results and full references of our AAAI 2023 work
When Neural Networks Fail to Generalize? A Model Sensitivity Perspective.
SAMix, an variant of SADA is accepted by MICCAI-2023!
Please check SAMix.
The figure above summarizes our algorithm comparisons framework, Spectral Adversarial Data Augmentation.
- First, our method computes model sensitivity map for the ERM model.
- Then, we run SADA to generate the augmented data and train the model with the JS-div regularization.
- Python 3.8
- PyTorch 1.8.1+
- A computing device with GPU
- (Not necessary) Install Anaconda3
- Install CUDA 11.0+
- Install PyTorch 1.8.1+
Noted that our code is tested based on PyTorch 1.8.1
All datasets used in our work are publicly available. In our experiments, we followed the data precrossing in the RandConv.
Train and evaluate an ERM model by
python ./AAAIcodeSubmissoin__model_sensitivity_map/train_ERM.py
The model will be saved in ./AAAIcodeSubmissoin__model_sensitivity_map/save_dir
.
Measure the sensitivity map of an ERM model by
python ./AAAIcodeSubmissoin__model_sensitivity_map/model_sensitivity_map.py
If you want to try SADA directly, a example of model sensitivity map of an ERM model trained on DIGTS dataset is provided as in
./AAAIcodeSubmissoin__model_sensitivity_map/sample/sample_map.pth
Train and evaluate the models with SADA by
python ./AAAIcodeSubmissoin__SADA/train_SADA.py
The key SADA data augmentation module is in
./AAAIcodeSubmissoin__SADA/sada.py
Augmentation settings for all datasets:
--epsilon
iteration of the checkpoint to load. #Default: 0.2--step_size
step size of the adversarial attack on the amplitude spectrum. #Default: 0.08--num_steps
batch size of the attack steps. #Default: 5--tau
settings for the early stop acceleration. #Default: 1--random_init
if or not initializing amplitude spertrums with random perturbations. #Default: True--randominit_type
type of random init type. #Default: 'normal'--criterion
loss functions to attack. #Default: 'ce', choices=['ce', 'kl']--direction
neg: standard attack || pos:inverse attack. #Default: neg, choices=['pos','neg']
Please cite these papers in your publications if it helps your research:
@inproceedings{zhang2023neural,
title={When neural networks fail to generalize? a model sensitivity perspective},
author={Zhang, Jiajin and Chao, Hanqing and Dhurandhar, Amit and Chen, Pin-Yu and Tajer, Ali and Xu, Yangyang and Yan, Pingkun},
booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
volume={37},
number={9},
pages={11219--11227},
year={2023}
}
We directly adopt some codes from the previous works as below:
We would like to thank these authors for sharing their codes.