Width-Adjusted-Regularization

[tabrisweapon]

Paper: will be uploaded soon

Venue: {if applicable, the venue where the paper appeared}

Dataset and threat model: CIFAR-10, l-inf, eps=8/255

Code: Training with WAR based on implementation of RST, for testing, please refer to https://github.com/tabrisweapon/A-temp-project

'''
python auto_cifar10.py --width=15 --model-dir=highest.pt
'''

Pre-trained model: https://www.dropbox.com/s/89uuo4w2iaitw04/highest.pt?dl=0

Log file: {link to log file of the evaluation}

Additional data: yes

Clean and robust accuracy: clean: 85.60%, PGD 20 * 0.003: 64.86%

Architecture: WRN-34-15

Description of the model/defense: A new training principle: stronger regularization for wider models

[fra31]

Hi,

thanks for the submission! I'll let you know once I've run the evaluation!

[tabrisweapon]

Hi,

thanks for the submission! I'll let you know once I've run the evaluation!

Thanks!

[fra31]

I got

clean accuracy: 85.60%
robust accuracy 59.78%

for eps=8/255. Is this consistent with your experiments?

[tabrisweapon]

I got

clean accuracy: 85.60%
robust accuracy 59.78%

for eps=8/255. Is this consistent with your experiments?

My evaluation has a little bit higher score. I got robust accuracy of 60.34% for eps=0.031, which is slightly lower than 8/255. Could this be a problem? I'm re-running the testing for eps=8/255 right now and it could take one or two days.

Following your instructions, my testing code can be found in https://github.com/tabrisweapon/A-temp-project/blob/master/auto_cifar10.py, and the key part is in line 76-77.
Could you kindly tell me if my implementation is correct?

[fra31]

I think something around 0.5% of difference between the robust accuracy at eps=0.031 and eps=8/255 is usual, so it should be fine.
Your implementation looks good to me, I just noticed that you used the older version of AA (now the standard version contains the targeted versions APGD-DLR and FAB and should be a bit faster, see also the README for details), which could lead to slightly higher values.

[tabrisweapon]

I think something around 0.5% of difference between the robust accuracy at eps=0.031 and eps=8/255 is usual, so it should be fine.
Your implementation looks good to me, I just noticed that you used the older version of AA (now the standard version contains the targeted versions APGD-DLR and FAB and should be a bit faster, see also the README for details), which could lead to slightly higher values.

Thanks for your help!
Knowing the difference of the epsilon parameter, We are re-training our method under eps=8/255 and will report our results later. We will also give the new version of AA a try.

[fra31]

I used eps=8/255 because that's indicated in the form above, but in the list there are a few models using eps=0.031, just we flag it,
Anyway, let me know when you have the new model!

[tabrisweapon]

I used eps=8/255 because that's indicated in the form above, but in the list there are a few models using eps=0.031, just we flag it,
Anyway, let me know when you have the new model!

Hi, my evaluation of eps=8/255 on our submitted model comes out and has a robust accuracy of 59.88%, very close to your evaluation. Since we are the highest result so far, we hope to release it for now and maybe report better results in the future. Our paper will also be uploaded in a few days. Could you please put our score onto the table?

[fra31]

I'm happy to add your results! I'd need to know how to cite your work on the list.
Also, in my opinion it is preferable to have a paper explaining how the model has been trained. If you can assure that your paper will be uploaded soon (within a few days), I'd link to this page so there is some info about the model from the form above (I'd suggest to add a list of authors). Could this work?

[tabrisweapon]

I'm happy to add your results! I'd need to know how to cite your work on the list.
Also, in my opinion it is preferable to have a paper explaining how the model has been trained. If you can assure that your paper will be uploaded soon (within a few days), I'd link to this page so there is some info about the model from the form above (I'd suggest to add a list of authors). Could this work?

Hi! Sorry for replying late. We've uploaded our paper, and you can find it at http://arxiv.org/abs/2010.01279. Please cite our work with this link too. Thanks a lot!

[fra31]

Hi, I've updated the list. Thanks again for the submission!

[tabrisweapon]

Hi, I've updated the list. Thanks again for the submission!

Thanks for your efforts of providing such an effective platform!