added SimBA (Simple Black-box Adversarial Attacks)

[kztakemoto]

Description

added SimBA

added code for SimBA (Simple Black-box Adversarial Attacks) implemented in Keras: art/attacks/evasion/simba.py. This is related to Issue #51 .

modified UniversalPerturbation (art/attacks/evasion/universal_perturbation.py ) to use SimBA in UniversalPerturbation as an attacker.

added a targeted attack version of UniversalPerturbation

added code for targeted attack version of UniversalPerturbation (TargetedUniversalPerturbation): art/attacks/evasion/targeted_universal_perturbation.py (see also https://github.com/hkthirano/targeted_UAP_CIFAR10). Note that FGSM and SimBA are only available as attackers in this implementation. This is related to Issue #393 .

modified UniversalPerturbation

to be able to give associated correct labels (i.e., y) because the current version of UniversalPerturbation only use the model predictions as the correct labels. This is related to Issue #393 .

Fixes # (issue)

Type of change

Please check all relevant options.

• Bug fix (non-breaking)
• New feature (non-breaking)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

Testing

I have tested this implementation with some examples: https://github.com/kztakemoto/SimBA-Keras

Test Configuration:

• OS: macOS Catalina version 10.15.5
• Python version 3.7.7
• ART version: 1.1.0
• Keras (v 2.3.1)

Checklist

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

[kztakemoto]

Hi @beat-buesser Thanks for your review again. According to your suggestions, I

• added the 2 tests to the file run_tests.sh after line 57.
• modified art/attacks/evasion/simba.py to run in PyTorch (thanks for the information about self.estimator.channels_first!).
• added PyTorch tests into tests/attacks/test_simba.py.
• run black on all modified files to fix PEP8 format and line lengths as follows: black --line-length 120 [path-to-file].
• confirmed that all modified files run correctly.

We are really happy that you are planning to list our paper in your wiki page!

[beat-buesser]

Hi @kztakemoto Thank you very much!

The new tests are now successfully running on Travis and the LGTM analysis does not raise any alerts.

The PEP8 check has found two more warnings that black does not fix, one about a too long line in a docstring and one for a lambda assignment. I have made suggestions above on how to resolve them.

I think after these two changes all the checks should be passing.

[kztakemoto]

@beat-buesser Thanks again. All your suggestions were accepted. We hope all the checks are passing.

[beat-buesser]

@kztakemoto I'm sorry, my last suggestion lead to another PEP8 warning, this last suggestion above fixes it.

[kztakemoto]

@beat-buesser thanks for your kind cooperation!!

[codecov]

Codecov Report

Merging #469 into dev_1.4.0 will increase coverage by 0.03%.
The diff coverage is 69.02%.

@@              Coverage Diff              @@
##           dev_1.4.0     #469      +/-   ##
=============================================
+ Coverage      67.39%   67.42%   +0.03%     
=============================================
  Files            132      134       +2     
  Lines          11212    11505     +293     
  Branches        1887     1963      +76     
=============================================
+ Hits            7556     7757     +201     
- Misses          2982     3042      +60     
- Partials         674      706      +32     

Impacted Files	Coverage Δ
art/attacks/evasion/simba.py	60.86% <60.86%> (ø)
art/attacks/evasion/universal_perturbation.py	89.41% <83.33%> (-0.95%)	⬇️
...attacks/evasion/targeted_universal_perturbation.py	88.09% <88.09%> (ø)

[beat-buesser]

Hi @kztakemoto Thank you very much for your contribution which is adding your Targeted Universal Perturbation attack and the Simple Black-box Adversarial Attacks attack to ART and for your work and patience during the review process!

[kztakemoto]

@beat-buesser Thank you very much. We are really happy to contribute to the development of ART!