CW efficiency improvement and bug fix, add CW binary search version, early stop PGD version, support L0 and Linf for CW and CWBS, rewrite FAB attack, fix MI-FGSM bug, rewrite JSMA.

.github/workflows/build_coverage.yml

@@ -23,36 +23,61 @@ jobs:
       matrix:
         os:
           - ubuntu-latest
-        python-version: ["3.7", "3.8", "3.9", "3.10", "3.11"]
+        python-version: ["3.7", "3.8", "3.9", "3.10", "3.11", "3.12"]
         # Bugs: 3.10 will become 3.1 if without quotes -> https://github.com/actions/setup-python/issues/695
         # For ubuntu 22.04: https://raw.githubusercontent.com/actions/python-versions/main/versions-manifest.json
-        pytorch-version: ["1.9.1", "1.10.1", "1.11.0", "1.12.1", "1.13.1", "2.0.1"]
+        pytorch-version: ["1.9.1", "1.10.1", "1.11.0", "1.12.1", "1.13.1", "2.0.1", "2.1.2", "2.2.0", "2.3.1"]
           # 1.5.1, 1.4.0 Model load error in robustbench.
           # 1.8.1, 1.7.1, 1.6.0 'padding==same' error in TIFGSM
         exclude:
           # https://github.com/pytorch/vision#installation
+          # pytorch 2.3 support python from 3.8 to 3.12
+          - pytorch-version: "2.3.1"
+            python-version: "3.7"
+          # pytorch 2.2 support python from 3.8 to 3.11
+          - pytorch-version: "2.2.0"
+            python-version: "3.7"
+          - pytorch-version: "2.2.0"
+            python-version: "3.12"
+          # pytorch 2.1 support python from 3.8 to 3.11
+          - pytorch-version: "2.1.2"
+            python-version: "3.7"
+          - pytorch-version: "2.1.2"
+            python-version: "3.12"
           # pytorch 2.0 support python from 3.8 to 3.11
           - pytorch-version: "2.0.1"
             python-version: "3.7"
+          - pytorch-version: "2.0.1"
+            python-version: "3.12"
           # pytorch 1.13 support python from 3.7.2 to 3.10
           - pytorch-version: "1.13.1"
             python-version: "3.11"
+          - pytorch-version: "1.13.1"
+            python-version: "3.12"
           # pytorch 1.12 support python from 3.7 to 3.10
           - pytorch-version: "1.12.1"
             python-version: "3.11"
+          - pytorch-version: "1.12.1"
+            python-version: "3.12"
           # pytorch 1.11 support python from 3.7 to 3.10
           - pytorch-version: "1.11.0"
             python-version: "3.11"
+          - pytorch-version: "1.11.0"
+            python-version: "3.12"
           # pytorch 1.10 support python from 3.6 to 3.9
           - pytorch-version: "1.10.1"
             python-version: "3.10"
           - pytorch-version: "1.10.1"
             python-version: "3.11"
+          - pytorch-version: "1.10.1"
+            python-version: "3.12"
           # pytorch 1.9 support python from 3.6 to 3.9
           - pytorch-version: "1.9.1"
             python-version: "3.10"
           - pytorch-version: "1.9.1"
             python-version: "3.11"
+          - pytorch-version: "1.9.1"
+            python-version: "3.12"
 
     runs-on: ${{ matrix.os }}
 

.gitignore

@@ -1,28 +1,28 @@
+.vscode/
 __pycache__
-.ipynb_checkpoints/*
+.ipynb_checkpoints/
 debug.log
 
-build/*
-_build
-dist/*
-torchattacks.egg-info/*
-data/*
-models/*
+build/
+dist/
+torchattacks.egg-info/
+data/
 */.*
 MENIFEST.in
 setup.cfg
 _commit.bat
+code_coverage/data/
 
-autoattack/*
+autoattack/
+.pytest_cache/
 
 demo/_*
-demo/data/*
-demo/models/*
-demo/torchdefenses/*
-demo/robustbench/*
-demo/autoattack/*
+demo/data
+demo/models
+demo/torchdefenses
+demo/autoattack
+
 TODO.txt
-.vscode/
 coverage.xml
 .coverage
 black.ipynb

README.md

@@ -94,10 +94,10 @@ pip install -e .
         ```
     * By label
         ```python
-        atk.set_mode_targeted_by_label(quiet=True)
         # shift all class loops one to the right, 1=>2, 2=>3, .., 9=>0
         target_labels = (labels + 1) % 10
-        adv_images = atk(images, target_labels)
+        atk.set_mode_targeted_by_label(target_labels=target_labels, quiet=True)
+        adv_images = atk(images, labels)
         ```
     * Return to default
         ```python
@@ -128,12 +128,6 @@ pip install -e .
         atk2 = torchattacks.PGD(model, eps=8/255, alpha=2/255, iters=40, random_start=True)
         atk = torchattacks.MultiAttack([atk1, atk2])
         ```
-    * Binary search for CW
-        ```python
-        atk1 = torchattacks.CW(model, c=0.1, steps=1000, lr=0.01)
-        atk2 = torchattacks.CW(model, c=1, steps=1000, lr=0.01)
-        atk = torchattacks.MultiAttack([atk1, atk2])
-        ```
     * Random restarts
         ```python
         atk1 = torchattacks.PGD(model, eps=8/255, alpha=2/255, iters=40, random_start=True)
@@ -160,7 +154,7 @@ The distance measure in parentheses.
 |     **EOTPGD**<br />(Linf)      | Comment on "Adv-BNN: Improved Adversarial Defense through Robust Bayesian Neural Network" ([Zimmermann, 2019](https://arxiv.org/abs/1907.00895))          | [EOT](https://arxiv.org/abs/1707.07397)+PGD                                                                            |
 |    **APGD**<br />(Linf, L2)     | Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks ([Croce et al., 2020](https://arxiv.org/abs/2001.03994)) |                                                                                                                        |
 |    **APGDT**<br />(Linf, L2)    | Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks ([Croce et al., 2020](https://arxiv.org/abs/2001.03994)) | Targeted APGD                                                                                                          |
-|   **FAB**<br />(Linf, L2, L1)   | Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attack ([Croce et al., 2019](https://arxiv.org/abs/1907.02044))                    |                                                                                                                        |
+|   **AFAB**<br />(Linf, L1, L2)   | Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attack ([Croce et al., 2019](https://arxiv.org/abs/1907.02044))                 |                                                                             |
 |   **Square**<br />(Linf, L2)    | Square Attack: a query-efficient black-box adversarial attack via random search ([Andriushchenko et al., 2019](https://arxiv.org/abs/1912.00049))         |                                                                                                                        |
 | **AutoAttack**<br />(Linf, L2)  | Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks ([Croce et al., 2020](https://arxiv.org/abs/2001.03994)) | APGD+APGDT+FAB+Square                                                                                                  |
 |     **DeepFool**<br />(L2)      | DeepFool: A Simple and Accurate Method to Fool Deep Neural Networks ([Moosavi-Dezfooli et al., 2016](https://arxiv.org/abs/1511.04599))                   |                                                                                                                        |
@@ -181,8 +175,15 @@ The distance measure in parentheses.
 | **EADEN**<br />(L1, L2) | EAD: Elastic-Net Attacks to Deep Neural Networks ([Chen, Pin-Yu, et al., 2018](https://arxiv.org/abs/1709.04114))                 | :heart_eyes: Contributor [Riko Naka](https://github.com/rikonaka)                               |
 | **PIFGSM (PIM)**<br />(Linf) | Patch-wise Attack for Fooling Deep Neural Network ([Gao, Lianli, et al., 2020](https://arxiv.org/abs/2007.06765))                 | :heart_eyes: Contributor [Riko Naka](https://github.com/rikonaka)                               |
 | **PIFGSM++ (PIM++)**<br />(Linf) | Patch-wise++ Perturbation for Adversarial Targeted Attacks ([Gao, Lianli, et al., 2021](https://arxiv.org/abs/2012.15503))                 | :heart_eyes: Contributor [Riko Naka](https://github.com/rikonaka)                               |
-
-
+| **CWL0**<br />(L0) | Towards Evaluating the Robustness of Neural Networks ([Carlini N, Wagner D, 2017](https://arxiv.org/abs/1608.046443))                 | :heart_eyes: Contributor [Riko Naka](https://github.com/rikonaka)                               |
+| **CWLinf**<br />(Linf) | Towards Evaluating the Robustness of Neural Networks ([Carlini N, Wagner D, 2017](https://arxiv.org/abs/1608.046443))                 | :heart_eyes: Contributor [Riko Naka](https://github.com/rikonaka)                               |
+| **CWBSL0 (Binary Search Version)**<br />(L0) | Towards Evaluating the Robustness of Neural Networks ([Carlini N, Wagner D, 2017](https://arxiv.org/abs/1608.046443))                 | :heart_eyes: Contributor [Riko Naka](https://github.com/rikonaka)                               |
+| **CWBSL2 (Binary Search Version)**<br />(L2) | Towards Evaluating the Robustness of Neural Networks ([Carlini N, Wagner D, 2017](https://arxiv.org/abs/1608.046443))                 | :heart_eyes: Contributor [Riko Naka](https://github.com/rikonaka)                               |
+| **CWBSLinf (Binary Search Version)**<br />(Linf) | Towards Evaluating the Robustness of Neural Networks ([Carlini N, Wagner D, 2017](https://arxiv.org/abs/1608.046443))                 | :heart_eyes: Contributor [Riko Naka](https://github.com/rikonaka)                               |
+| **ESPGD (Early-Stopped PGD Version)**<br />(Linf) | Attacks Which Do Not Kill Training Make Adversarial Learning Stronger ([Zhang, Jingfeng, 2020](https://arxiv.org/abs/2002.11242))                 | :heart_eyes: Contributor [Riko Naka](https://github.com/rikonaka)                               |
+|   **FAB**<br />(Linf)   | Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attack ([Croce et al., 2019](https://arxiv.org/abs/1907.02044))                    | :heart_eyes: Contributor [Riko Naka](https://github.com/rikonaka)                       |
+|   **FABL1**<br />(L1)   | Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attack ([Croce et al., 2019](https://arxiv.org/abs/1907.02044))                    | :heart_eyes: Contributor [Riko Naka](https://github.com/rikonaka)                       |
+|   **FABL2**<br />(L2)   | Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attack ([Croce et al., 2019](https://arxiv.org/abs/1907.02044))                    | :heart_eyes: Contributor [Riko Naka](https://github.com/rikonaka)                       |
 
 ## :bar_chart: Performance Comparison
 

code_coverage/script/pickle_cifar10.py

@@ -0,0 +1,25 @@
+import torch
+from torchvision import datasets, transforms
+
+transform_test = transforms.Compose([
+    transforms.ToTensor(),
+])
+testset = datasets.CIFAR10(
+    root='../data', train=False, download=True, transform=transform_test)
+test_loader = torch.utils.data.DataLoader(
+    testset, batch_size=10, shuffle=False)
+
+
+def split(testloader):
+    for (x, y) in testloader:
+        torch.save(x, 'images.pth')
+        torch.save(y, 'labels.pth')
+        break
+
+
+def main():
+    split(test_loader)
+
+
+if __name__ == '__main__':
+    main()

code_coverage/script/resnet.py

@@ -0,0 +1,117 @@
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+
+class BasicBlock(nn.Module):
+    expansion = 1
+
+    def __init__(self, in_planes, planes, stride=1):
+        super(BasicBlock, self).__init__()
+        self.conv1 = nn.Conv2d(in_planes, planes, kernel_size=3, stride=stride, padding=1, bias=False)
+        self.bn1 = nn.BatchNorm2d(planes)
+        self.conv2 = nn.Conv2d(planes, planes, kernel_size=3, stride=1, padding=1, bias=False)
+        self.bn2 = nn.BatchNorm2d(planes)
+        self.relu = nn.ReLU()
+
+        self.shortcut = nn.Sequential()
+        if stride != 1 or in_planes != self.expansion * planes:
+            self.shortcut = nn.Sequential(
+                nn.Conv2d(in_planes, self.expansion * planes, kernel_size=1, stride=stride, bias=False),
+                nn.BatchNorm2d(self.expansion * planes)
+            )
+
+    def forward(self, x):
+        out = self.relu(self.bn1(self.conv1(x)))
+        out = self.bn2(self.conv2(out))
+        out += self.shortcut(x)
+        out = F.relu(out)
+        return out
+
+
+class Bottleneck(nn.Module):
+    expansion = 4
+
+    def __init__(self, in_planes, planes, stride=1):
+        super(Bottleneck, self).__init__()
+        self.conv1 = nn.Conv2d(in_planes, planes, kernel_size=1, bias=False)
+        self.bn1 = nn.BatchNorm2d(planes)
+        self.conv2 = nn.Conv2d(planes, planes, kernel_size=3, stride=stride, padding=1, bias=False)
+        self.bn2 = nn.BatchNorm2d(planes)
+        self.conv3 = nn.Conv2d(planes, self.expansion * planes, kernel_size=1, bias=False)
+        self.bn3 = nn.BatchNorm2d(self.expansion * planes)
+        self.relu = nn.ReLU()
+
+        self.shortcut = nn.Sequential()
+        if stride != 1 or in_planes != self.expansion * planes:
+            self.shortcut = nn.Sequential(
+                nn.Conv2d(in_planes, self.expansion * planes, kernel_size=1, stride=stride, bias=False),
+                nn.BatchNorm2d(self.expansion * planes)
+            )
+
+    def forward(self, x):
+        out = self.relu(self.bn1(self.conv1(x)))
+        out = self.relu(self.bn2(self.conv2(out)))
+        out = self.bn3(self.conv3(out))
+        out += self.shortcut(x)
+        out = F.relu(out)
+        return out
+
+
+class ResNet(nn.Module):
+    def __init__(self, block, num_blocks, num_classes=10):
+        super(ResNet, self).__init__()
+        self.in_planes = 64
+
+        self.conv1 = nn.Conv2d(3, 64, kernel_size=3, stride=1, padding=1, bias=False)
+        self.bn1 = nn.BatchNorm2d(64)
+        self.layer1 = self._make_layer(block, 64, num_blocks[0], stride=1)
+        self.layer2 = self._make_layer(block, 128, num_blocks[1], stride=2)
+        self.layer3 = self._make_layer(block, 256, num_blocks[2], stride=2)
+        self.layer4 = self._make_layer(block, 512, num_blocks[3], stride=2)
+        self.linear = nn.Linear(512 * block.expansion, num_classes)
+
+    def _make_layer(self, block, planes, num_blocks, stride):
+        strides = [stride] + [1] * (num_blocks - 1)
+        layers = []
+        for stride in strides:
+            layers.append(block(self.in_planes, planes, stride))
+            self.in_planes = planes * block.expansion
+        return nn.Sequential(*layers)
+
+    def forward(self, x):
+        out = F.relu(self.bn1(self.conv1(x)))
+        out = self.layer1(out)
+        out = self.layer2(out)
+        out = self.layer3(out)
+        out = self.layer4(out)
+        out = F.avg_pool2d(out, 4)
+        out = out.view(out.size(0), -1)
+        out = self.linear(out)
+        return out
+
+
+def ResNet18(num_classes=10):
+    return ResNet(BasicBlock, [2, 2, 2, 2])
+
+
+def ResNet34():
+    return ResNet(BasicBlock, [3, 4, 6, 3])
+
+
+def ResNet50():
+    return ResNet(Bottleneck, [3, 4, 6, 3])
+
+
+def ResNet101():
+    return ResNet(Bottleneck, [3, 4, 23, 3])
+
+
+def ResNet152():
+    return ResNet(Bottleneck, [3, 8, 36, 3])
+
+
+def test():
+    net = ResNet18()
+    y = net(torch.randn(1, 3, 32, 32))
+    print(y.size())