Unable to Access Triggered Dataset in BadNet Attack

[pheobyy]

I would like to be able to access and inspect the dataset containing the added trigger after running the BadNet attack.

I ran the attack using the following command:
python examples/backdoor_attack.py --color --verbose 1 --pretrained --validate_interval 1 --epochs 10 --lr 0.01 --mark_random_init --attack badnet --save
After running the attack, I expected to be able to access the dataset containing the trigger, but I'm facing some difficulties in doing so. Here are the issues I encountered:

• I tried to inspect the CIFAR-10 dataset, assuming that the trigger would be added to the data. However, it appears that the dataset remains unchanged. (I printed a few images with true label 0)
• I attempted to look at the trigger image by adding the following code to the method add_mark in abstract.py (https://github.com/ain-soph/trojanzoo/blob/main/trojanvision/attacks/abstract.py) :
  backdoored_img = self.mark.add_mark(x, **kwargs) plt.imsave("trigger_img.jpg", backdoored_img.cpu().numpy())
  But this seems to contain only the trigger but not the entire image with the trigger.
• I also tried running the attack in "dataset" train mode, as follows:
  python examples/backdoor_attack.py --color --verbose 1 --pretrained --validate_interval 1 --epochs 10 --lr 0.01 --mark_random_init --attack badnet --train_mode dataset --save
  But I'm facing the same issues.

[ain-soph]

• train_mode == 'batch' won't modify dataset, it's attaching the trigger on demand during get_data by calling add_mark method
  Related codes:

  trojanzoo/trojanvision/attacks/abstract.py

  Lines 114 to 132 in d5085fd

  	case 'batch':
  	loader = self.dataset.get_dataloader(
  	'train', batch_size=self.dataset.batch_size)
  	return self.model._train(epochs, loader_train=loader,
  	get_data_fn=self.get_data,
  	**kwargs)
  	case 'dataset':
  	mix_dataset = torch.utils.data.ConcatDataset([self.dataset.loader['train'].dataset,
  	self.poison_set])
  	loader = self.dataset.get_dataloader('train', dataset=mix_dataset)
  	return self.model._train(epochs, loader_train=loader, **kwargs)
  	case 'loss':
  	if 'loss_fn' in kwargs.keys():
  	kwargs['loss_fn'] = functools.partial(self.loss_weighted, loss_fn=kwargs['loss_fn'])
  	else:
  	kwargs['loss_fn'] = self.loss_weighted
  	return self.model._train(epochs, **kwargs)
  	case _:
  	raise NotImplementedError(f'{self.train_mode=}')

• When train_mode == 'batch', calling add_mark(_input) (where _input is the clean batch image) shall return the entire image with the trigger.

  If you want to access the trigger image only, you need to view attack.mark.mark, which is a [4, H, W] tensor and the last channel is the transparency following RGBA color channel standard.
  Related docs: https://ain-soph.github.io/trojanzoo/trojanvision/marks.html#trojanvision.marks.Watermark

  You also mentioned calling add_mark will only return image contain the trigger. I'm confused why it is the case. Please show a brief snippet and its running results. A possible reason might be your trigger size is 32*32, which is the same as image size and it overwrites all pixels.

• When train_mode == 'dataset', there will be a new property called poison_set.
  Related codes:
  

  trojanzoo/trojanvision/attacks/abstract.py

  Line 106 in d5085fd

  self.poison_set = self.get_poison_dataset()

  
  Related docs:
  https://ain-soph.github.io/trojanzoo/trojanvision/attacks/index.html#trojanvision.attacks.BackdoorAttack

[ain-soph]

Btw, just a trick, using torchvision.utils.save_image(backdoored_img, "trigger_img.jpg") is easier than plt.imsave("trigger_img.jpg", backdoored_img.cpu().numpy())

https://pytorch.org/vision/stable/generated/torchvision.utils.save_image.html?highlight=save_#torchvision.utils.save_image

[pheobyy]

Thank you for your quick response! :)

I tried it again with train_mode == 'dataset' and it worked

For train_mode == 'batch' :
I tried using torchvision.utils.save_image(backdoored_img, "trigger_img.jpg") instead of plt.imsave and everything worked fine! The "trigger_img" contained the images with the trigger attached.
So the issue seems to be only related to plt.imsave.
This is how "trigger_img" looks like if I use plt.imsave :

[ain-soph]

This doesn't seem to be a trigger either. I think plt.imsave requires HWC rather than CHW order (not sure, never use it).
So you might only output 3-pixel width somehow.

[ain-soph]

I guess your problem has been solved in previous discussion. I'll close this issue. Feel free to re-open if there is still any problem related to this issue.