problem with metattack

[shanzhiq]

Thanks for this wonderful repo. However, when I run the metattack example ,the result is not promising
Here is my result when attack Cora with metattack
Training...
100/100 [====================] - Total: 520.68ms - 5ms/step- loss: 0.0713 - acc: 0.996 - val_loss: 0.574 - val_acc: 0.847
Evaluating...
1/1 [====================] - Total: 2.01ms - 2ms/step- loss: 0.522 - acc: 0.847
Before attack
╒═════════╤═══════════╕
│ Names │ Objects │
╞═════════╪═══════════╡
│ loss │ 0.521524 │
├─────────┼───────────┤
│ acc │ 0.846579 │
╘═════════╧═══════════╛
Peturbing graph...: 100%|██████████| 253/253 [01:00<00:00, 4.17it/s]Evaluating...
1/1 [====================] - Total: 2.08ms - 2ms/step- loss: 0.528 - acc: 0.844
After evasion attack
╒═════════╤═══════════╕
│ Names │ Objects │
╞═════════╪═══════════╡
│ loss │ 0.528431 │
├─────────┼───────────┤
│ acc │ 0.844064 │
╘═════════╧═══════════╛
Training...
32/100 [=====>..............] - ETA: 0s- loss: 0.212 - acc: 0.956 - val_loss: 0.634 - val_acc: 0.807
100/100 [====================] - Total: 407.58ms - 4ms/step- loss: 0.0601 - acc: 0.996 - val_loss: 0.704 - val_acc: 0.787
Evaluating...
1/1 [====================] - Total: 1.66ms - 1ms/step- loss: 0.711 - acc: 0.819
After poisoning attack
╒═════════╤═══════════╕
│ Names │ Objects │
╞═════════╪═══════════╡
│ loss │ 0.710625 │
├─────────┼───────────┤
│ acc │ 0.818913 │
╘═════════╧═══════════╛

[EdisonLeeeee]

Thank you for reporting. There are some reasons for the poor results:

• Randomness. Current implementation is unstable and you need to run it multiple times with different random seeds to get a decent performance.
• Validation with model picking. As mentioned here, you can disable the validation step to improve the poisoning attack performance:

# Disable the validation step and the model picking
trainer.fit(attacker.data(), mask=splits.train_nodes)