Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add link to diffusers library #7

Merged
merged 2 commits into from Nov 12, 2022
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
54 changes: 52 additions & 2 deletions readme.md
Expand Up @@ -5,6 +5,7 @@ This repo is the official PyTorch implementation for the paper [Pseudo Numerical

by Luping Liu, Yi Ren, Zhijie Lin, Zhou Zhao (Zhejiang University).


## What does this code do?
This code is not only the official implementation for PNDM, but also a generic framework for DDIM-like models including:
- [x] [Pseudo Numerical Methods for Diffusion Models on Manifolds (PNDM)](https://openreview.net/forum?id=PlKWVd2yBkY)
Expand All @@ -25,6 +26,55 @@ supported by this code and the role of each object.
All of them can be combined at will, so this code provide at least 5x3x4=60 choices to generate samples.


## Integration with 馃 Diffusers library

PNDM is now also available in 馃Ж Diffusers and accesible via the [PNDMPipeline](https://huggingface.co/docs/diffusers/api/pipelines/pndm).
Diffusers allows you to test PNDM in PyTorch in just a couple lines of code.

You can install diffusers as follows:

```
pip install diffusers torch accelerate
```

And then try out the sampler/scheduler with just a couple lines of code:

```python
from diffusers import PNDMPipeline

model_id = "google/ddpm-cifar10-32"

# load model and scheduler
pndm = PNDMPipeline.from_pretrained(model_id)

# run pipeline in inference (sample random noise and denoise)
image = pndm(num_inference_steps=50).images[0]

# save image
image.save("pndm_generated_image.png")
```

The PNDM scheduler can also be used with more powerful diffusion models such as [Stable Diffusion](https://huggingface.co/docs/diffusers/v0.7.0/en/api/pipelines/stable_diffusion#stable-diffusion-pipelines)

You simply need to [accept the license on the Hub](https://huggingface.co/runwayml/stable-diffusion-v1-5), login with `huggingface-cli login` and install transformers:

```
pip install transformers
```

Then you can run:

```python
from diffusers import StableDiffusionPipeline, PNDMScheduler

pndm = PNDMScheduler.from_config("runwayml/stable-diffusion-v1-5", subfolder="scheduler")
pipeline = StableDiffusionPipeline.from_pretrained("runwayml/stable-diffusion-v1-5", scheduler=pndm)

image = pipeline("An astronaut riding a horse.").images[0]
image.save("astronaut_riding_a_horse.png")
```


## How to run the code

### Dependencies
Expand Down Expand Up @@ -70,6 +120,6 @@ If you find the code useful for your research, please consider citing:
```
This work is built upon some previous papers which might also interest you:
- Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems 33 (2020): 6840-6851.
- Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising Diffusion Implicit Models. International Conference on Learning Representations. 2020.
- Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-Based Generative Modeling through Stochastic Differential Equations. International Conference on Learning Representations. 2020.
- Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising Diffusion Implicit Models. International Conference on Learning Representations. 2021.
- Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-Based Generative Modeling through Stochastic Differential Equations. International Conference on Learning Representations. 2021.