The Code Repository for "CCDM: Continuous Conditional Diffusion Models for Image Generation"
This repository provides the source codes for the experiments in our papers for CCDMs.
If you use this code, please cite
@misc{ding2024ccdm,
title={{CCDM}: Continuous Conditional Diffusion Models for Image Generation},
author={Xin Ding and Yongwei Wang and Kao Zhang and Z. Jane Wang},
year={2024},
eprint={2405.03546},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
Illustration of the CCGM task with sample images from the UTKFace and Steering Angle datasets.
The overall workflow of CCDMs. The training process entails adapting a modified U-Net using the proposed hard vicinal loss and integrating CcGANs' label embedding network, referred to as ILI. For the sampling process, a classifier-free guidance-based mechanism is employed to produce a linear combination of a conditional output and an unconditional output from the trained U-Net. A DDIM-based sampler is then used to generate new images based on this linear combination.
The network architecture of the denoising U-Net.
Here, we provide a list of crucial software environments and python packages employed in the conducted experiments. Please note that we use different computational platforms for our experiments.
For computing NIQE scores and implementing the NIQE filtering (Support both Windows and Linux):
| Item | Version | Item | Version |
|---|---|---|---|
| OS | Win11 | OS | Linux |
| Python | 3.11.3 | Python | 3.10.12 |
| Matlab | 2023a | Matlab | 2021b |
For implementing CCDM (Support both Windows and Linux):
| Item | Version | Item | Version |
|---|---|---|---|
| OS | Win11 | OS | Linux |
| Python | 3.11.3 | Python | 3.10.12 |
| CUDA | 11.8 | CUDA | 12.1 |
| numpy | 1.23.5 | numpy | 1.26.4 |
| torch | 2.0.1 | torch | 2.2.1 |
| torchvision | 0.15.2 | torchvision | 0.17.1 |
| Pillow | 9.5.0 | Pillow | 9.0.1 |
| accelearate | 0.20.3 | accelearate | 0.27.2 |
For implementing ReACGAN, ADCGAN, ADM-G, and CFG (Support Windows):
| Item | Version |
|---|---|
| OS | Win11 |
| Python | 3.11.3 |
| CUDA | 11.8 |
| numpy | 1.23.5 |
| torch | 2.0.1 |
| torchvision | 0.15.2 |
| Pillow | 9.5.0 |
| accelearate | 0.20.3 |
| wandb | 0.15.7 |
For implementing CcGAN and Dual-NDA (Support Linux):
| Item | Version |
|---|---|
| OS | Linux |
| Python | 3.9 |
| CUDA | 11.4 |
| numpy | 1.23.0 |
| torch | 1.12.1 |
| torchvision | 0.13.1 |
| Pillow | 8.4.0 |
| accelearate | 0.18.0 |
We use the preprocessed datasets provided by Ding et. al. (2023).
Download the following h5 file and put it in ./datasets/RC-49.
RC-49_64x64_OneDrive_link
RC-49_64x64_BaiduYun_link
Download the following h5 files and put them in ./datasets/UTKFace.
UTKFace_64x64_Onedrive_link
UTKFace_64x64_BaiduYun_link
UTKFace_128x128_OneDrive_link
UTKFace_128x128_BaiduYun_link
UTKFace_192x192_OneDrive_link
UTKFace_192x192_BaiduYun_link
Download the following h5 files and put them in ./datasets/SteeringAngle.
SteeringAngle_64x64_OneDrive_link
SteeringAngle_64x64_BaiduYun_link
SteeringAngle_128x128_OneDrive_link
SteeringAngle_128x128_BaiduYun_link
Download the following h5 file and put it in ./datasets/Cell-200.
Cell-200_64x64_OneDrive_link
Cell-200_64x64_BaiduYun_link
Please download the zip file from either OneDrive or BaiduYun and extract its contents to the ./CCDM directory. This zip archive includes the necessary checkpoints for the ILI's embedding networks, as well as the corresponding checkpoints for the evaluation models related to each individual experiment.
Following Ding et. al. (2023) and Ding et. al. (2024), distinct training codes have been provided for various datasets.
For simplicity, we only show how to implement the proposed CCDM in each experiment.
Go to the directory ./RC-49/RC-49_64x64/CCGM/CCDM. Run the training script ./scripts/run_train_ccdm.bat for Windows or ./scripts/run_train_ccdm.sh for Linux. Please correctly set ROOT_PATH, DATA_PATH, EVAL_PATH, and NIQE_PATH.
Please note that we also provide the training scripts named run_train_ccdpm.bat and run_train_ccdpm.sh for implementing CcDPM on RC-49.
Go to the directory ./UTKFace/UK64/CCGM/CCDM. Run the training script ./scripts/run_train.bat for Windows or ./scripts/run_train.sh for Linux. Please correctly set ROOT_PATH, DATA_PATH, EVAL_PATH, and NIQE_PATH.
Go to the directory ./UTKFace/UK128/CCGM/CCDM. Run the training script ./scripts/run_train.bat for Windows or ./scripts/run_train.sh for Linux. Please correctly set ROOT_PATH, DATA_PATH, EVAL_PATH, and NIQE_PATH.
Go to the directory ./UTKFace/UK192/CCGM/CCDM. Run the training script ./scripts/run_train.bat for Windows or ./scripts/run_train.sh for Linux. Please correctly set ROOT_PATH, DATA_PATH, EVAL_PATH, and NIQE_PATH.
Go to the directory ./UTKFace/SA64/CCGM/CCDM. Run the training script ./scripts/run_train.bat for Windows or ./scripts/run_train.sh for Linux. Please correctly set ROOT_PATH, DATA_PATH, EVAL_PATH, and NIQE_PATH.
Go to the directory ./UTKFace/SA128/CCGM/CCDM. Run the training script ./scripts/run_train.bat for Windows or ./scripts/run_train.sh for Linux. Please correctly set ROOT_PATH, DATA_PATH, EVAL_PATH, and NIQE_PATH.
Go to the directory ./Cell-200/Cell-200_64x64/CCGM/CCDM. Run the training script ./scripts/run_train.bat for Windows or ./scripts/run_train.sh for Linux. Please correctly set ROOT_PATH, DATA_PATH, EVAL_PATH, and NIQE_PATH.
For all methods except ReACGAN and ADCGAN, after the training, the evaluation usually automatically starts. The evaluation setups are consistent with Ding et. al. (2023).
In the bash scripts for training each method, enable --dump_fake_for_NIQE --niqe_dump_path <YOUR_NIQE_PATH> to dump fake images for computing NIQE scores. Please note that you need set the <YOUR_NIQE_PATH> correctly. Fake images for computing NIQE are usually stored in ./CcGAN_TPAMI_NIQE/fake_data. Run the batch script run_test.bat to computing the average NIQE scores.