awesome-diffusion

Introduction

Models based on diffusion has shown fantastic performance on image generation and other tasks. Although each day comes several papers about diffusion, there are several major topics about it :applications , theory and engineering improvement, conditional generation.

Newest Survey Papers

Year	Title	Venue	Paper	Code
2023	Diffusion Models: A Comprehensive Survey of Methods and Applications	Arxiv	Link	/

In this survey of diffusion models, more than 300 papers were surveyed, but only few months after it was published, another 300 papers has been produced,especially after Controlnet was released.

Papers

applications

mainly about 3d diffusion applications and cross-modality applications, which are both very popular topics nowadays.

Time	Title	Venue	Code
20220407	Video Diffusion Models		link
20221222	Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation	arxiv	link
2023	MM-Diffusion: Learning Multi-Modal Diffusion Models for Joint Audio and Video Generation	CVPR	link
2023	VideoFusion: Decomposed Diffusion Models for High-Quality Video Generation	CVPR	link
20230323	Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators	arxiv	link
20230522	VDT: An Empirical Study on Video Diffusion with Transformers	arxiv	link
20220615	Diffusion Models for Video Prediction and Infilling	arxiv	link
20221123	Unsupervised Learning for Physical Interaction through Video Prediction	arxiv	link
2023	DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation	CVPR	link
2023	ERNIE-ViLG 2.0: Improving Text-to-Image Diffusion Model with Knowledge-Enhanced Mixture-of-Denoising-Experts	CVPR	link
20230512	One Transformer Fits All Distributions in Multi-Modal Diffusion at Scale	arxiv	link
2023	DreamFusion: Text-to-3D using 2D Diffusion	ICLR oral	link
20221120	Auto Regressive latent diffusion model	arxiv	[link](Synthesizing Coherent Story with Auto-Regressive Latent Diffusion Models)
20230531	Control4D: Dynamic Portrait Editing by Learning 4D GAN from 2D Diffusion-based Editor	arxiv	link
20230413	RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignment	arxiv	link
20230419	Anything-3D: Towards Single-view Anything Reconstruction in the Wild	arxiv	link

Conditional generation

this part include text2image, few-shot , one-short and other researches about conditional generation. Generating images and videos under certain type of instructions has a prosperous future in commercial and daily uses. At the same time , this can be very difficult to satisfy people’s expectations.

Time	Title	Venue	Code
2022128	Refining Generative Process with Discriminator Guidance in Score-based Diffusion Models	arxiv	link
20210112	D2C: Diffusion-Denoising Models for Few-shot Conditional Generation	arxiv	link
20220623	Entropy-driven Sampling and Training Scheme for Conditional Diffusion Generation	arxiv	link
20220829	Frido: Feature Pyramid Diffusion for Complex Scene Image Synthesis	arxiv	link
20230221	Diffusion Models and Semi-Supervised Learners Benefit Mutually with Few Labels	arxiv	link
20211126	Conditional Image Generation with Score-Based Diffusion Models	arxiv	link
20230519	Late-Constraint Diffusion Guidance for Controllable Image Synthesis	arxiv	link
20230503	Shap-E: Generating Conditional 3D Implicit Functions-openai	arxiv	[link](openai/shap-e: Generate 3D objects conditioned on text or images (github.com))

Theory and Engineering improvement

In fact, while diffusion models have a higher performance upbound, the training and inferring cost can be high. So it would be very interesting and meaningful to research about reducing unnecessary cost of diffusion-based models.

One of the most famous paper that fit this part is DDPM,which made application possible for diffusion models, but it is to old and famous too be presented in this chart.

Time	Title	Venue	Code
20220101	Elucidating the Design Space of Diffusion-Based Generative Models	arxiv	link
20210218	Improved Denoising Diffusion Probabilistic Models-openai	arxiv	link
2022	Vector Quantized Diffusion Model for Text-to-Image Synthesis	CVPR	link

dataset

some new dataset for diffusion models.

dataset	paper	github
Diversify Your Vision Datasets with Automatic Diffusion-Based Augmentation	link	link
DiffuseExpand: Expanding dataset for 2D medical image segmentation using diffusion models	link	link
Realistic Data Enrichment for Robust Image Segmentation in Histopathology	link	/
A Multi-Institutional Open-Source Benchmark Dataset for Breast Cancer Clinical Decision Support using Synthetic Correlated Diffusion Imaging Data	link
Diffusion-based Data Augmentation for Skin Disease Classification: Impact Across Original Medical Datasets to Fully Synthetic Images	link
Diversify Your Vision Datasets with Automatic Diffusion-Based Augmentation	link	link