Make ImageNet-C module pip-installable

[nottombrown]

We're interested in using the ImageNet-C transformations as a baseline attack in the Unrestricted Adversarial Examples Challenge: openphilanthropy/unrestricted-adversarial-examples#40

It would be easier for us to do this if the module included a setup.py and was available on pypi

[hendrycks]

I will look into this in a week (after ICLR's submission deadline).

[nottombrown]

Thanks! Good luck with the the submission

[hendrycks]

This package is pip installable.

I have a question about the warm-up challenge. Imagine we pre-specify large-scale content that we would like a network to detect. So consider a network that classifies 1 if the large-scale input contains the content of the pre-specified image, and 0 if the input contains completely different content. The network must only be robust over one image's content, rather than be robust across a broad distribution of images such as birds and bikes. Do you know if current approaches can reliably defend against unrestricted attacks (or stronger attacks such as PGD \ell_\infty with \epsilon=16/255) for a single pre-specified large-scale image? The reason I ask is because some papers suggest adversarial robustness requires the number of training samples to be exponential in the image dimension.

[nottombrown]

Awesome! Thanks for the change! Interesting question: I believe that adversarial training will find a robust model if (a) you know what attacks the attacker will use, (b) you know which inputs are in the test set, (c) your model has enough capacity, and (d) you train for long enough. An example of this is in Adversarial Spheres by Gilmer et al. https://arxiv.org/abs/1801.02774 For their high-dimensional spheres task, they find that if you train the model against PGD for long enough, it will eventually learn a decision boundary that cleanly separates the two spheres and makes no misclassifications. (cc @jmgilmer)

…

On Tue, Oct 2, 2018 at 11:03 PM Dan Hendrycks ***@***.***> wrote: This package <https://github.com/hendrycks/robustness/tree/master/ImageNet-C/imagenet_c> is pip installable. I have a question about the warm-up challenge. Imagine we pre-specify large-scale content that we would like a network to detect. So consider a network that classifies 1 if the large-scale input contains the content of the pre-specified image, and 0 if the input contains completely different content. The network must only be robust over one image's content, rather than be robust across a broad distribution of content such as birds and bikes. Do you know if current approaches can reliably defend against unrestricted attacks (or stronger attacks such as PGD \ell_\infty with \epsilon=16/255) for a single pre-specified large-scale image? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#2 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AASt3wA0lPi-5Spg946FqdilG8eQsgxZks5uhFMUgaJpZM4WynLS> .

[jmgilmer]

Adversarial spheres wasn't really intended to be a difficult dataset. So it's not clear to me whether or not we can draw conclusions about perfect models for the sphere dataset to perfect models for image data.

The "semantic ball" around a single image is going to be a very complex object, probably much more so than a sphere. There it's plausible to me that we might need an exponential amount of data (haven't had the time yet to read the paper Dan is referring to).

Regarding the success of adversarial training, https://arxiv.org/abs/1804.11285 found that adversarial training on CIFAR-10 for long enough with a large enough network would yield a model that was 100% robust on the training set, but that this robustness did not generalize to the test set (they were only 50% robust at l_infinity eps=8/255 and 0% robust in larger threat models). So I don't think a solution is as simple as just adversarially train longer with a larger network.

[nottombrown]

Thanks for expanding, Justin. It seems like you agree with the following statement? I believe that adversarial training will find a robust model *if all of the following hold true,* (a) you can easily enumerate all the attacks the attacker will use, (b) you know which inputs are in the test set, (c) your model has enough capacity, and (d) you train for long enough.

…

On Thu, Oct 4, 2018 at 12:06 AM Justin Gilmer ***@***.***> wrote: Adversarial spheres wasn't really intended to be a difficult dataset, the fact that we were able to obtain perfect models on the spheres isn't really indicative in either direction about the question Dan is asking. The "semantic ball" around a single image is going to be a very complex object, probably much more so than a sphere. There it's plausible to me that we might need an exponential amount of data (haven't had the time yet to read the paper Dan is referring to). Regarding the success of adversarial training, https://arxiv.org/abs/1804.11285 found that adversarial training on CIFAR-10 for long enough with a large enough network would yield a model that was 100% robust on the training set, but that this robustness did not generalize to the test set (they were only 50% robust at l_infinity eps=8/255 and 0% robust in larger threat models). So I don't think a solution is as simple as just adversarially train longer with a larger network. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#2 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AASt316K7d22IGexv78ts-5L4W92zPwcks5uhbONgaJpZM4WynLS> .

[jmgilmer]

If the goal is l_p robustness to images in the training set then adversarial training long enough could get you there (or you could alternatively use a NN classifier). But if you want significant l_p robustness to new images in the test set, that has yet to be demonstrated on CIFAR-10 for even modest sized epsilon.

So I guess my agreement depends on the question :).

[nottombrown]

Cool, I agree with that.