Is that normal that poisoned predictions do not have the same shape as the true labels?

[EvilWatermelon]

I try to get the confusion matrix of the poisoned predictions but the poisoned predictions have a different shape[0] as the original test labels.

not_target = np.logical_not(np.all(y_test == targets, axis=1))
px_test, py_test = backdoor.poison(X_test[not_target], y_test[not_target])
poison_preds = np.argmax(model.predict(px_test), axis=1)

cm = confusion_matrix(labels, poison_preds )
fp = cm.sum(axis=0) - np.diag(cm)
fn = cm.sum(axis=1) - np.diag(cm)
tp = np.diag(cm)
tn = cm.sum() - (fp + fn + tp)

(11970,) = poison_preds and the true labels are over 12k.

[beat-buesser]

Hi @EvilWatermelon I think they should not have a different shape. What is the shape of your X_test and y_test?

[EvilWatermelon]

The not target is the same as in the clean label backdoor notebook example. It checks for logical not between y_test and the chosen target. It seems that it checks all test labels.

[beat-buesser]

Would you be able to share a longer example script with more details of the setups? Maybe one that I can run?

[EvilWatermelon]

I took the code from https://github.com/Trusted-AI/adversarial-robustness-toolbox/blob/main/notebooks/poisoning_attack_clean_label_backdoor.ipynb with another dataset. The code is basiclly the same. You should get the same problem when executing this notebook.

You can try to execute my code here https://github.com/EvilWatermelon/Risk-Measurement-Framework/blob/main/risk_measurement_framework/rmf/case_study.py

Note: this framework is not useable for other ML models and only works with my case study. My problem starts on line 304.

I use https://www.kaggle.com/datasets/meowmeowmeowmeowmeow/gtsrb-german-traffic-sign this dataset

[beat-buesser]

I have not been able to observe a similar behavior in poisoning_attack_clean_label_backdoor.ipynb using ART 1.10.3.

Which version of ART are you using?

Does it also happen if you change the parameters slightly so that not_target does not select all samples?

[EvilWatermelon]

After trying to understand the not_target logic I think I figured out why the number of labels is different. The logical_not checks all False values and saves them as True and vice versa. This happens for all True values (labels that are equals the target) in an array on axis 1. I think in summary this means a comparison, if y_test is a target, these labels will not be poisoned. That's why the px_test size is smaller than the original X_test.