Projected and Greedy Randomized Block Coordinate Descent (NeurIPS 2021) #5972
Conversation
for more information, see https://pre-commit.ci
Codecov Report
@@ Coverage Diff @@
## master #5972 +/- ##
==========================================
+ Coverage 85.30% 85.51% +0.20%
==========================================
Files 402 403 +1
Lines 21731 22024 +293
==========================================
+ Hits 18537 18833 +296
+ Misses 3194 3191 -3
📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
|
Thank you @wsad1 I am a bit confused, though, that you assigned me. Is there any action required from my side at the moment? |
|
@sigeisler No action required until reviews lands. This just means you are the author of this PR :) |
|
@sigeisler Thanks for adding these. Looks pretty good! This is also related to my current research and it did really help! |
Co-authored-by: Jintang Li <cnljt@outlook.com>
|
Hi @rusty1s I think we are done at the moment and would appreciate input from your side. |
|
Sounds good, thanks for the effort. |
examples/rbcd_attack_poisoning.py
Outdated
| print(f'Clean accuracy: {clean_accuracy:.3f}') | ||
|
|
||
|
|
||
| class PoisoningPRBCDAttack(PRBCDAttack): |
There was a problem hiding this comment.
Any reason this is implemented in the example rather than as a module in nn/models?
There was a problem hiding this comment.
The main reason is that the train loop can have so many different variations. Of course, we could pass a method that defines the training procedure instead of tons of configurations. Moreover, the implementation here requires an additional dependency (i.e., higher).
If you prefer to have the "poisoning" variant as a dedicated model or configurable option, I am more than happy to implement it.
|
@rusty1s thank you for your time and the comments! Here are my responses to the high-level questions:
This was in analogy to the (old) implementation of GNNExplainer (have not checked how it is now). But I agree that we could add it to the
The idea is to save (valuable) memory of the GPU/... Note that here we maintain a clean graph and construct its perturbed counterpart. Thus, without moving the clean clean graph to |
|
Thanks for the clarification. This makes sense to me. |
|
@rusty1s I worked through your comments and adapted the code accordingly. I left some comments to explain etc. Please let me know if there are further action items. Thanks again for your time and the helpful feedback! |
|
Hi @rusty1s I know you must be quite busy, however, I am wondering how to proceed. Thanks. |
|
This should be ready to merge. Will take a last look and merge it soon. |
|
Hi @rusty1s thanks for seeing this through :-) I have seen that you now have moved this to the Btw. if you need any input whatsoever for promoting this as one of the models in the new |
|
Hi @sigeisler. Yes, I indeed moved this to |
|
@rusty1s makes a lot of sense! Thanks for clarifying. |
Hi PyG-team,
Thank you for the tremendous effort in putting this code base together and maintaining it to this high level of quality.
Similar to the
ExplainerandGNN-Explainermodels, I hereby submit the code for our adversarial attacks based on Randomized Block Coordinate Descent (RBCD) that appeared (NeurIPS 2021). Our efficient (i.e sparse) gradient-based attacks apply to most models in PyG and cover the most popular threat models (see below for more details).Adversarial robustness is quite related to explainability (here considering w.r.t. graph structure). Specifically, in an adversarial attack, we are asking for a limited number of edges that harm the prediction the most, and (GNN-) Explainer seeks for the set of friendly edges (i.e. minimal set without changing prediction). Due to the similarities, I have integrated attacks similar to how GNNExplainer was integrated.
Adversarial robustness of GNNs is an emerging field (the two seminal works were cited more than 1000 times) and it would be of even greater practical relevance if the "entry barrier" would be lowered. Moreover, having practical adversarial attacks in PyGs code base comes with opportunities for its users. These include: (1) seamless adversarial training for PyG's large collection of models; (2) adversarial attacks can be understood as a building block towards counterfactual explainability (and explainability is already part of PyG's codebase); (3) Adversarial attacks can be used for unsupervised representation learning (similar to Deep Graph Infomax).
Importantly, having adversarial attacks generically implemented that can be directly applied to models (i.e. basic adaptive attacks) would counteract a common antipattern in the graph learning community. In our work
Are Defenses for Graph Neural Networks Robust? (NeurIPS 2022), we show that such adaptive attacks would greatly help to obtain more realistic robustness estimates.
Our Projected- and Greedy Randomized Block Coordinate Descent attacks are (afaik) the only practical general-purpose gradient-based adversarial attacks on GNNs (i.e. work with sparse matrices and come with a linear overhead for test time attacks). Our gradient-based approach relaxes the weights of the adjacency matrix from {0,1} -> [0,1] during the attack and, therefore, seamlessly integrates with all models that support weighted adjacency matrices and do not apply non-differentiable operations to the edge weights. Although, non-differentiable models can often be made differentiable (e.g. using the gumble softmax trick).
Also with respect to tasks and threat models, this implementation covers the most popular cases. The defaults are set for node and graph classification (see unit tests for the latter) and other tasks (e.g. regression) can be handled by providing an appropriate model loss pair. The threat models covered are: (a) "global attacks": the adversary targets a large fraction of nodes at once (e.g. the test set); (b) "local attacks": the adversary targets the prediction of a single node in node-classification; any combination with (I) "evasion": attack at test time (most relevant for inductive learning); (II) "poisoning": attack at training time (most relevant for transductive learning, see poisoning example).
I am happy to include further suggestions and answer any questions. Please let me know if you have any feedback.