Counterfactual Explanations for Face Forgery Detection Via Adversarial Removal of Artifacts

Training code for ICME 2024 paper Counterfactual Explanations for Face Forgery Detection Via Adversarial Removal of Artifacts.

Getting Started

Installation

• clone the repository:

git clone https://github.com/yangli-lab/Artifact-Eraser
cd Artifact-Eraser

conda create -n AR python=3.7
conda activate AR
pip install -r requirements.txt

Pretrained Models

Please download the pretrained models from the following links and save them in the 'Save_Folder'

Path	Save_Folder	Description
E4E Encoder	./pretrained_models/	Encoder4editing model pretrained by omertov.
StyleGAN	./pretrained_models/	StyleGAN model pretrained on FFHQ taken from rosinality with 1024x1024 output resolution.
IR-SE50 Model	./pretrained_models/	Pretrained IR-SE50 model taken from TreB1eN for use in ID loss during training.
MAT	./classifiers/multiple_attention/pretrained/	Pretrained deepfake detection model, MAT, by yoctta.
RECCE	./classifiers/RECCE/pretrained_models/	Pretrained deepfake detection model, RECCE, by VISION-SJTU.

Training

Training the e4e encoder

The training code of E4E encoder is borrowed from their official repository. To train the e4e encoder, make sure the paths to the required models, and prepare your dataset.

CUDA_VISIBLE_DEVICES=0 python ./scripts/train_finetune.py --dataset_type ffhq_encode \
--exp_dir ./exp/celebdf_decoder \
--start_from_latent_avg \
--use_w_pool \
--w_discriminator_lambda 0 \
--progressive_start 0 \
--train_encoder 0 \
--train_decoder 1 \
--lpips_lambda 1.0 \
--id_lambda 0.1 \
--l2_lambda 1.0 \
--val_interval 1000 \
--max_steps 1000 \
--image_interval 100 \
--board_interval 100 \
--stylegan_size 1024 \
--save_interval 500 \
--checkpoint_path /path/to/pretrained/e4e_model \
--workers 4 \
--batch_size 8 \
--test_batch_size 2 \
--test_workers 4 

Training Classifier(Optional)

You can use your pretrained deepfake classifier model or train a new one with the provided code, please refer to the classifiers folder. Let's take training MAT as an example

CUDA_VISIBLE_DEVICES=0 python3 ./classifiers/train_mat.py \
--dataset_name ffpp \
--root_dir ffpp_path \
--batch_size 32 \
--read_method frame_by_frame \
--lr 1e-3 \
--momentum 0.9 \
--weight_decay 1e-6 \
--max_iteration 1200 \
--max_warmup_iteration 0 \
--epochs 1000 \
--num_workers 2 \
--snapshot_frequency 100 \
--snapshot_template snapshot \
--exp_root save_folder \

Artifact Eraser Attack

Make sure you have specified a pretrained e4e model by --checkpoint_path and pretrained deepfake classifier by --classifier_ckpt, and run the code bellow. You can specify the attack strength and attack times by setting parameters ALPHA and EPOCH.

DATASET=celebdf
CLASSIFIER=efficient
ATTACK_METHOD=fgsm
EPOCH=50
FRAME=14
ALPHA=0.0002
NAME=attack_latent_nc_${DATASET}_${CLASSIFIER}_${ATTACK_METHOD}_${EPOCH}_${FRAME}_${ALPHA}
CUDA_VISIBLE_DEVICES=0 python3 ./attack/attack_celebdf.py \
--dataset ${DATASET} \
--stylegan_size 1024 \
--encoder_type Encoder4Editing \
--device cuda:0 \
--start_from_latent_avg True \
--classifier_ckpt classifier_path \
--classifier ${CLASSIFIER} \
--attack_method ${ATTACK_METHOD} \
--mask full \
--beta 0.02 \
--alpha ${ALPHA} \
--epoch ${EPOCH} \
--frames ${FRAME} \
--loss_type mse_loss \
--lpips_lambda 0 \
--lpips_type alex \
--id_lambda 0 \
--mse_lambda 0 \
--dataset_dir data_dir \
--label_file label_for_dfdc_dataset \
--video_level False \
--attack_num 50 \
--log_dir log_path/${NAME} \
2>&1 | tee log_path/${NAME}.log \

Acknowledgments

This code borrows heavily from e4e_encoder.

Citation

If you find our work interesting, please feel free to cite our paper:

put the citation here