Skip to content

[Docs] add note on caching in fast diffusion #8675

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jun 24, 2024
+6 −6
Merged

[Docs] add note on caching in fast diffusion #8675

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
28ef949
add note on caching in fast diffusion
sayakpaul Jun 24, 2024
1b4c4d4
formatting
sayakpaul Jun 24, 2024
3e3d102
Update docs/source/en/tutorials/fast_diffusion.md
sayakpaul Jun 24, 2024
6ca6fbd
Merge branch 'main' into add-caching-note
sayakpaul Jun 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions docs/source/en/tutorials/fast_diffusion.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,13 +34,10 @@ Install [PyTorch nightly](https://pytorch.org/) to benefit from the latest and f
pip3 install --pre torch --index-url https://download.pytorch.org/whl/nightly/cu121
```

<Tip>

The results reported below are from a 80GB 400W A100 with its clock rate set to the maximum. <br>
> [!TIP]
> The results reported below are from a 80GB 400W A100 with its clock rate set to the maximum.
> If you're interested in the full benchmarking code, take a look at [huggingface/diffusion-fast](https://github.com/huggingface/diffusion-fast).

If you're interested in the full benchmarking code, take a look at [huggingface/diffusion-fast](https://github.com/huggingface/diffusion-fast).

</Tip>

## Baseline

Expand Down Expand Up @@ -170,6 +167,9 @@ Using SDPA attention and compiling both the UNet and VAE cuts the latency from 3
<img src="https://huggingface.co/datasets/sayakpaul/sample-datasets/resolve/main/progressive-acceleration-sdxl/SDXL%2C_Batch_Size%3A_1%2C_Steps%3A_30_3.png" width=500>
</div>

> [!TIP]
> From PyTorch 2.3.1, you can control the caching behavior of `torch.compile()`. This is particularly beneficial for compilation modes like `"max-autotune"` which performs a grid-search over several compilation flags to find the optimal configuration. Learn more in the [Compile Time Caching in torch.compile](https://pytorch.org/tutorials/recipes/torch_compile_caching_tutorial.html) tutorial.

### Prevent graph breaks

Specifying `fullgraph=True` ensures there are no graph breaks in the underlying model to take full advantage of `torch.compile` without any performance degradation. For the UNet and VAE, this means changing how you access the return variables.
Expand Down