Adv-Diffusion: Imperceptible Adversarial Face Identity Attack via Latent Diffusion Model

Official code for Adv-Diffusion: Imperceptible Adversarial Face Identity Attack via Latent Diffusion Model (AAAI 2024).

Decheng Liu^*1, Xijun Wang^*1, Chunlei Peng^†1, Nannan Wang¹, Ruimin Hu¹, Xinbo Gao²

¹Xidian University, ²Chongqing University of Posts and Telecommunications

Abstract

Adversarial attacks involve adding perturbations to the source image to cause misclassification by the target model, which demonstrates the potential of attacking face recognition models. Existing adversarial face image generation methods still can’t achieve satisfactory performance because of low transferability and high detectability. In this paper, we propose a unified framework Adv-Diffusion that can generate imperceptible adversarial identity perturbations in the latent space but not the raw pixel space, which utilizes strong inpainting capabilities of the latent diffusion model to generate realistic adversarial images. Specifically, we propose the identity-sensitive conditioned diffusion generative model to generate semantic perturbations in the surroundings. The designed adaptive strength-based adversarial perturbation algorithm can ensure both attack transferability and stealthiness. Extensive qualitative and quantitative experiments on the public FFHQ and CelebA-HQ datasets prove the proposed method achieves superior performance compared with the state-of-the-art methods without an extra generative model training process.

Install

• Build environment

conda env create -f env.yaml

• Download checkpoints

  Pretrained LDM can be found here.

  We use IR152, IRSE50, FaceNet and MobileFace model checkpoints that provided by [AMT-GAN](CGCL-codes/AMT-GAN: The official implementation of our CVPR 2022 paper "Protecting Facial Privacy: Generating Adversarial Identity Masks via Style-robust Makeup Transfer". (github.com)). The google drive link they provide is [here](assets.zip - Google 云端硬盘).

  The face parsing model we use can be found here. And the model checkpoints they provide is here: [google drive].

  You need to create a directory named as pretrained_model and put the checkpoints into it.

• Download datasets

  In our experiment we use FFHQ and CelebA-HQ datasets for evaluation. Because we do not own the datasets, you need to download them yourself. And you can refer to CelebAMask-HQ for CelebA-HQ download.

  After download you need to select several target images and source images and put them in the correct path as following.

  ├─celeba-hq_sample
  │  ├─src
  │  └─target
  ├─configs
  │  └─stable-diffusion
  │      └─intel
  ├─FaceParsing
  │  └─networks
  ├─ffhq_sample
  │  ├─src
  │  └─target
  ├─fr_model
  ├─ldm
  │  ├─data
  │  ├─models
  │  │  └─diffusion
  │  │      └─dpm_solver
  │  └─modules
  │      ├─diffusionmodules
  │      ├─distributions
  │      ├─encoders
  │      ├─image_degradation
  │      └─midas
  │          └─midas
  └─pretrained_model
  

Usage

bash eval.sh

Citation

@inproceedings{liu2024adv,
  title={Adv-diffusion: imperceptible adversarial face identity attack via latent diffusion model},
  author={Liu, Decheng and Wang, Xijun and Peng, Chunlei and Wang, Nannan and Hu, Ruimin and Gao, Xinbo},
  booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
  volume={38},
  number={4},
  pages={3585--3593},
  year={2024}
}

If you have any questions, please contact xdu [DOT] xjwang [AT] gmail [DOT] com.