SecMI on Latent Diffusion Models

This codebase implements SecMI (repo) on conditioned generation, including fine-tuned Stable Diffusion (SD) and vanilla SD. This is built on diffuser-0.11.1. Please refer to here for environment configuration.

We have modified the following files in order to perform SecMI:

# Implement reverse in DDIM
./src/diffusers/schedulers/scheduling_ddim.py
# Save reverse and denoising intermediate results
./src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion.py

Dataset

Please download datasets from here and unzip it.

There are three datasets included: Pokemon, Laion (2.5k), and coco2017val (2.5k).

SecMI w/ Stable Diffusion fine-tuned over the Pokemon dataset

Checkpoint

Please download the Pokemon-fine-tuned SD from here and unzip it.

Script

Please refer to the following script or run sh scripts/secmi_ldm_pokemon.sh

CUDA_VISIBLE_DEVICES=0 python -m src.mia.secmi \
--dataset pokemon \
--dataset-root /path/to/datasets \
--ckpt-path /path/to/sd-pokemon-checkpoint

SecMI w/ vanilla Stable Diffusion over Laion (as member) and coco2017val (as non-member)

Script

Please refer to the following script or run sh scripts/secmi_sd_laion.sh

CUDA_VISIBLE_DEVICES=0 python -m src.mia.secmi \
--dataset laion \
--dataset-root /path/to/datasets \
--ckpt-path runwayml/stable-diffusion-v1-5

Reference

Please cite our paper if you feel this is helpful:

@InProceedings{duan2023are,
  title = {Are Diffusion Models Vulnerable to Membership Inference Attacks?},
  author = {Duan, Jinhao and Kong, Fei and Wang, Shiqi and Shi, Xiaoshuang and Xu, Kaidi},
  booktitle = {Proceedings of the 40th International Conference on Machine Learning},
  pages = {8717--8730},
  year = {2023}
}