Codes and dataset (DFDM) for Model Attribution of Face-swap Deepfake Videos [Paper]
We created a new dataset, named DFDM, with 6,450 Deepfake videos generated by different Autoencoder models. Specifically, five Autoencoder models with variations in encoder, decoder, intermediate layer, and input resolution, respectively, have been selected to generate Deepfakes based on the same input. We have first observed the visible but subtle visual differences among different Deepfakes, demonstrating the evidence of model attribution artifacts. Then we take Deepfakes model attribution as a multiclass classification task and propose a spatial and temporal attention based method to explore the differences among Deepfakes in DFDM dataset.
The DFDM dataset includes face-swap Deepfakes videos generated from five Autoencoder models based on Faceswap software, including the Faceswap, Lightweight, IAE, Dfaker, and DFL-H128. Three H.264 compression rates are considered to get videos with different qualities, including lossless with the constant rate factor (crf) as 0, high quality with crf as 10, and low quality with crf as 23. A total of 6,450 Deepfakes have been created.
Model | Input | Output | Encoder | Decoder | Variation |
---|---|---|---|---|---|
Faceswap (baseline) | 64 | 64 | 4Conv+1Ups | 3Ups+1Conv | / |
Lightweight | 64 | 64 | 3Conv+1Ups | 3Ups+1Conv | Encoder |
IAE | 64 | 64 | 4Conv | 4Ups+1Conv | Intermediate layers; Shared Encoder&Decoder |
Dfaker | 64 | 128 | 4Conv+1Ups | 4Ups+3Residual+1Conv | Decoder |
DFL-H128 | 128 | 128 | 4Conv+1Ups | 3Ups+1Conv | Input resolution |
The total size of the dataset files is ~82.4GB.
The database contains three folders "DFDM_crf*" with videos in three subfolders:
- DFDM_crf0 ~ 53.7G: 2,150 videos from 5 models with lossless qualities;
- DFDM_crf10 ~ 24.4G: 2,150 videos from 5 models with higher qualities;
- DFDM_crf23 ~ 4.3G: 2,150 videos from 5 models with lower qualities.
File name: id**_id**_(scene)_(model_name).mp4
To extract frames/faces, we recommend to use FFMPEG, please refer to the Faceswap USAGE.md.
If you would like to access the DFDM dataset, please fill out this google form. The download link will be sent to you once the form is accepted (in 72 hours). If you have any questions, please send email to [dfdmdataset@gmail.com].
We designed a simple and effective Deepfake video model attribution method based on Spatial and Temporal Attention (DMA-STA), and achieved an overall accuracy of ~70% in identifying the higher-quality Deepfakes in DFDM dataset.
- Prerequisites: our code requires PyTorch and Python 3.
- Pre-trained model: download the pretrained ResNet-50 model, put it into the 'data' folder, i.e., 'data/resnet50-19c8e357.pth'.
- Data structure: DF_class*/Video*/Frame1.png, ... (cropped face images).
- Hyper-parameters: you can modify 'config.py' to set more detailed hyper-parameters.
The DFDM database is released only for academic research. Researchers from educational institute are allowed to use this database freely for noncommercial purpose.
If you use this dataset, please cite the following paper:
@inproceedings{jia2022model,
title={Model attribution of face-swap deepfake videos},
author={Jia, Shan and Li, Xin and Lyu, Siwei},
booktitle={2022 IEEE International Conference on Image Processing (ICIP)},
pages={2356--2360},
year={2022},
organization={IEEE}
}