Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement SIGN-OPT: A Query-Efficient Hard-label Black-box Attack #1730

Merged
merged 26 commits into from
Jun 27, 2022

Conversation

synergit
Copy link
Contributor

@synergit synergit commented Jun 4, 2022

Description

Implement SIGN-OPT: A Query-Efficient Hard-label Black-box Attack

Fixes #1331;

Please note that this PR is a replacement for #1695 by resolving the double signed commit issue

Type of change

Please check all relevant options.

  • Improvement (non-breaking)
  • Bug fix (non-breaking)
  • New feature (non-breaking)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

Testing

Please describe the tests that you ran to verify your changes. Consider listing any relevant details of your test configuration.

  • unit test
  • performance test

Test Configuration:

  • OS: Mac and Google Colab
  • Python version:3.8
  • ART version or commit number
  • PyTorch

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

Signed-off-by: synergit <syner.wang@gmail.com>
Signed-off-by: synergit <syner.wang@gmail.com>
Signed-off-by: synergit <syner.wang@gmail.com>
Signed-off-by: synergit <syner.wang@gmail.com>
@beat-buesser beat-buesser marked this pull request as ready for review June 4, 2022 19:06
@beat-buesser beat-buesser self-assigned this Jun 4, 2022
@beat-buesser beat-buesser added the enhancement New feature or request label Jun 4, 2022
@beat-buesser beat-buesser added this to Pull request open in ART 1.11.0 via automation Jun 4, 2022
@beat-buesser beat-buesser added this to the ART 1.11.0 milestone Jun 4, 2022
@beat-buesser beat-buesser marked this pull request as draft June 4, 2022 19:07
@beat-buesser beat-buesser self-requested a review June 4, 2022 19:08
@beat-buesser beat-buesser changed the title Developmnet issue 1331 v3 Implement SIGN-OPT: A Query-Efficient Hard-label Black-box Attack Jun 4, 2022
@codecov-commenter
Copy link

codecov-commenter commented Jun 4, 2022

Codecov Report

Merging #1730 (f7573e6) into dev_1.11.0 (3f106d7) will decrease coverage by 15.10%.
The diff coverage is 9.74%.

Impacted file tree graph

@@               Coverage Diff               @@
##           dev_1.11.0    #1730       +/-   ##
===============================================
- Coverage       88.15%   73.05%   -15.11%     
===============================================
  Files             259      262        +3     
  Lines           21411    21770      +359     
  Branches         3805     3888       +83     
===============================================
- Hits            18874    15903     -2971     
- Misses           1591     4949     +3358     
+ Partials          946      918       -28     
Impacted Files Coverage Δ
art/attacks/evasion/sign_opt.py 9.42% <9.42%> (ø)
art/attacks/evasion/__init__.py 100.00% <100.00%> (ø)
art/attacks/evasion/pe_malware_attack.py 14.28% <0.00%> (-84.80%) ⬇️
art/attacks/poisoning/gradient_matching_attack.py 8.64% <0.00%> (-80.08%) ⬇️
...feature_adversaries/feature_adversaries_pytorch.py 14.01% <0.00%> (-78.51%) ⬇️
art/attacks/evasion/dpatch.py 12.96% <0.00%> (-78.40%) ⬇️
art/attacks/evasion/boundary.py 14.45% <0.00%> (-78.32%) ⬇️
..._air_flickering/over_the_air_flickering_pytorch.py 17.16% <0.00%> (-77.62%) ⬇️
...attacks/evasion/geometric_decision_based_attack.py 13.65% <0.00%> (-77.57%) ⬇️
...efences/trainer/adversarial_trainer_fbf_pytorch.py 16.49% <0.00%> (-76.29%) ⬇️
... and 78 more

@lgtm-com
Copy link

lgtm-com bot commented Jun 4, 2022

This pull request introduces 2 alerts when merging c50ce77 into 3f106d7 - view on LGTM.com

new alerts:

  • 1 for Module is imported with 'import' and 'import from'
  • 1 for Variable defined multiple times

Signed-off-by: synergit <syner.wang@gmail.com>
Signed-off-by: synergit <syner.wang@gmail.com>
Signed-off-by: synergit <syner.wang@gmail.com>
@lgtm-com
Copy link

lgtm-com bot commented Jun 9, 2022

This pull request introduces 2 alerts when merging 46a0424 into 3f106d7 - view on LGTM.com

new alerts:

  • 1 for Module is imported with 'import' and 'import from'
  • 1 for Variable defined multiple times

Signed-off-by: synergit <syner.wang@gmail.com>
@lgtm-com
Copy link

lgtm-com bot commented Jun 13, 2022

This pull request introduces 2 alerts when merging 5041e79 into 3f106d7 - view on LGTM.com

new alerts:

  • 1 for Module is imported with 'import' and 'import from'
  • 1 for Variable defined multiple times

@synergit synergit marked this pull request as ready for review June 14, 2022 02:10
Co-authored-by: Beat Buesser <49047826+beat-buesser@users.noreply.github.com>
Copy link
Collaborator

@beat-buesser beat-buesser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @synergit Thank you very much for your first PR to ART contributing the Sign-Opt evasion attack! I have added a few review comments on small details. Let me know what you think!

ART 1.11.0 automation moved this from Pull request open to Pull request review Jun 15, 2022
synergit and others added 5 commits June 15, 2022 17:53
Co-authored-by: Beat Buesser <49047826+beat-buesser@users.noreply.github.com>
Co-authored-by: Beat Buesser <49047826+beat-buesser@users.noreply.github.com>
Co-authored-by: Beat Buesser <49047826+beat-buesser@users.noreply.github.com>
Co-authored-by: Beat Buesser <49047826+beat-buesser@users.noreply.github.com>
Co-authored-by: Beat Buesser <49047826+beat-buesser@users.noreply.github.com>
@lgtm-com
Copy link

lgtm-com bot commented Jun 15, 2022

This pull request introduces 2 alerts when merging 4668f50 into 3f106d7 - view on LGTM.com

new alerts:

  • 1 for Module is imported with 'import' and 'import from'
  • 1 for Variable defined multiple times

synergit and others added 4 commits June 18, 2022 18:45
Co-authored-by: Beat Buesser <49047826+beat-buesser@users.noreply.github.com>
Signed-off-by: synergit <syner.wang@gmail.com>
Signed-off-by: synergit <syner.wang@gmail.com>
@lgtm-com
Copy link

lgtm-com bot commented Jun 18, 2022

This pull request introduces 2 alerts when merging 151f09f into 3f106d7 - view on LGTM.com

new alerts:

  • 1 for Module is imported with 'import' and 'import from'
  • 1 for Variable defined multiple times

Signed-off-by: synergit <syner.wang@gmail.com>
Signed-off-by: synergit <syner.wang@gmail.com>
Signed-off-by: synergit <syner.wang@gmail.com>
Signed-off-by: synergit <syner.wang@gmail.com>
Signed-off-by: synergit <syner.wang@gmail.com>
@lgtm-com
Copy link

lgtm-com bot commented Jun 21, 2022

This pull request introduces 2 alerts when merging a42ddff into 313f373 - view on LGTM.com

new alerts:

  • 1 for Module is imported with 'import' and 'import from'
  • 1 for Variable defined multiple times

Signed-off-by: synergit <syner.wang@gmail.com>
@lgtm-com
Copy link

lgtm-com bot commented Jun 22, 2022

This pull request introduces 2 alerts when merging e65e04b into e442608 - view on LGTM.com

new alerts:

  • 1 for Module is imported with 'import' and 'import from'
  • 1 for Variable defined multiple times

Signed-off-by: synergit <syner.wang@gmail.com>
@lgtm-com
Copy link

lgtm-com bot commented Jun 23, 2022

This pull request introduces 2 alerts when merging f7573e6 into e442608 - view on LGTM.com

new alerts:

  • 1 for Module is imported with 'import' and 'import from'
  • 1 for Variable defined multiple times

Copy link
Collaborator

@beat-buesser beat-buesser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @synergit Thank you very much and congratulations for your first pull request to ART contributing the Sign-Opt black-box evasion attack!

@beat-buesser beat-buesser merged commit 2ac2cbd into Trusted-AI:dev_1.11.0 Jun 27, 2022
ART 1.11.0 automation moved this from Pull request review to Pull request done Jun 27, 2022
@synergit synergit deleted the developmnet_issue_1331_v3 branch August 9, 2022 22:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
No open projects
ART 1.11.0
  
Pull request done
Development

Successfully merging this pull request may close these issues.

None yet

3 participants