TRADES adversarial training protocol

[Zaid-Hameed]

Description

TRADES is an important adversarial training approach because it provides a better tradeoff between robustness against adversarial attacks and clean accuracy. TRADES has been proposed in paper "Theoretically Principled Trade-off between Robustness and Accuracy".

Paper link: https://proceedings.mlr.press/v97/zhang19p.html

It is also standard component of recent and more advanced adversarial training approaches.

Fixes #2031

Type of change

Please check all relevant options.

• Improvement (non-breaking)
• Bug fix (non-breaking)
• New feature (non-breaking)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

Testing

Please describe the tests that you ran to verify your changes. Consider listing any relevant details of your test configuration.

• Trades adversarial training implementation produces results similar to original implementation
• All functions in implemented code work as expected

Test Configuration:

• OS: Red Hat Enterprise Linux 8.7 (Ootpa)
• Python version: 3.9.12
• ART version or commit number: 1.14.1
• TensorFlow / Keras / PyTorch / MXNet version: PyTorch 1.13.1+cu117

Checklist

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

[codecov-commenter]

Codecov Report

Merging #2131 (3fe948d) into dev_1.15.0 (bbb92cf) will increase coverage by 12.24%.
The diff coverage is 89.76%.

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

@@               Coverage Diff               @@
##           dev_1.15.0    #2131       +/-   ##
===============================================
+ Coverage       73.39%   85.63%   +12.24%     
===============================================
  Files             297      299        +2     
  Lines           26521    26648      +127     
  Branches         4864     4878       +14     
===============================================
+ Hits            19464    22821     +3357     
+ Misses           5930     2580     -3350     
- Partials         1127     1247      +120     

Impacted Files	Coverage Δ
...nces/trainer/adversarial_trainer_trades_pytorch.py	87.73% <87.73%> (ø)
art/defences/trainer/__init__.py	100.00% <100.00%> (ø)
art/defences/trainer/adversarial_trainer_trades.py	100.00% <100.00%> (ø)

... and 65 files with indirect coverage changes

[beat-buesser]

Hi @Zaid-Hameed Thank you very much for your pull request! I have a few minor review comments, what do you think?

[beat-buesser]

Hi @Zaid-Hameed Thank you very much and congratulations to your first PR merged to ART!

[Zaid-Hameed]

Thanks a lot @beat-buesser for all the help in it and the review. :)