white box attack for auto-attack

[JIA-HONG-CHU]

hello , I want to ask what is the auto-attack setting in the white box, because the result of running the default value in vit_small_patch16_224 is different from the result of your paper.

my command,
python3 white_box_test.py --data_dir imagenet --mode auto --model vit_small_patch16_224

my result,
sample size is : 1000
clean accuracy: 73.8 %
Model vit_small_patch16_224 robust accuracy for AutoAttack perturbations with
Linf norm ≤ 0.001 : 28.4 %
Linf norm ≤ 0.003 : 1.2 %
Linf norm ≤ 0.005 : 0.0 %
Linf norm ≤ 0.008 : 0.0 %
Linf norm ≤ 0.01 : 0.0 %
Linf norm ≤ 0.1 : 0.0 %

tks for reply.

[RulinShao]

Hi @JIA-HONG-CHU,

Thanks for your interest in our work! We used the default setting of AutoAttack to run the experiments, but they may have changed the default setting and updated a stronger version of AutoAttack as you can check here, which may yield different scores compared with our previous results. Other factors like the random seed can also cause slight differences. But our key point is to compare the robustness of different models and we expect the AutoAttack to be the most strong attack in current literature. In this case, we recommend you run similar experiments on CNNs and compare the relative performances. Hope it helps!

[JIA-HONG-CHU]

ok tks