From a5e3c8b30eebe7651e1f33129bacd4a5908ab908 Mon Sep 17 00:00:00 2001 From: regisss Date: Wed, 30 Nov 2022 21:11:29 +0100 Subject: [PATCH 1/4] Add doc for Stable Diffusion on Habana Gaudi --- docs/source/_toctree.yml | 2 ++ docs/source/optimization/habana.mdx | 56 +++++++++++++++++++++++++++++ 2 files changed, 58 insertions(+) create mode 100644 docs/source/optimization/habana.mdx diff --git a/docs/source/_toctree.yml b/docs/source/_toctree.yml index 957144488323..354d381cd7e3 100644 --- a/docs/source/_toctree.yml +++ b/docs/source/_toctree.yml @@ -49,6 +49,8 @@ title: "OpenVINO" - local: optimization/mps title: "MPS" + - local: optimization/habana + title: "Habana Gaudi" title: "Optimization/Special Hardware" - sections: - local: training/overview diff --git a/docs/source/optimization/habana.mdx b/docs/source/optimization/habana.mdx new file mode 100644 index 000000000000..64fd16a7ff49 --- /dev/null +++ b/docs/source/optimization/habana.mdx @@ -0,0 +1,56 @@ + + +# How to use Stable Diffusion on Habana Gaudi + +🤗 Diffusers is compatible with Habana Gaudi through 🤗 [Optimum Habana](https://huggingface.co/docs/optimum/habana/usage_guides/stable_diffusion). + +## Requirements + +- Optimum Habana 1.3 or later, [here](https://huggingface.co/docs/optimum/habana/installation) is how to install it. +- SynapseAI 1.7. + + +## Inference Pipeline + +To generate images with Stable Diffusion 1 and 2 on Gaudi, you need to instantiate two instances: +- A pipeline with [`GaudiStableDiffusionPipeline`](https://huggingface.co/docs/optimum/habana/package_reference/stable_diffusion_pipeline). This pipeline supports *text-to-image generation*. +- A scheduler with [`GaudiDDIMScheduler`](https://huggingface.co/docs/optimum/habana/package_reference/stable_diffusion_pipeline#optimum.habana.diffusers.GaudiDDIMScheduler). This scheduler has been optimized for Habana Gaudi. + +When initializing the pipeline, you have to specify `use_habana=True` to deploy it on HPUs. +Furthermore, in order to get the fastest possible generations you should enable **HPU graphs** with `use_hpu_graphs=True`. +Finally, you will need to specify a [Gaudi configuration](https://huggingface.co/docs/optimum/habana/package_reference/gaudi_config) which can be downloaded from the H[ugging Face Hub](https://huggingface.co/Habana). + +```python +from optimum.habana import GaudiConfig +from optimum.habana.diffusers import GaudiDDIMScheduler, GaudiStableDiffusionPipeline +model_name = "stabilityai/stable-diffusion-2-base" +scheduler = GaudiDDIMScheduler.from_pretrained(model_name, subfolder="scheduler") +pipeline = GaudiStableDiffusionPipeline.from_pretrained( + model_name, + scheduler=scheduler, + use_habana=True, + use_hpu_graphs=True, + gaudi_config="Habana/stable-diffusion", +) +``` + +You can then call the pipeline to generate images by batches from one or several prompts: +```python +outputs = pipeline( + prompt=["High quality photo of an astronaut riding a horse in space", "Face of a yellow cat, high resolution, sitting on a park bench"], + num_images_per_prompt=10, + batch_size=4, +) +``` + +For more information, check out Optimum Habana's [documentation](https://huggingface.co/docs/optimum/habana/usage_guides/stable_diffusion) and the [example](https://github.com/huggingface/optimum-habana/tree/main/examples/stable-diffusion) provided in the official Github repository. From a4adc024ec2a38260e9b2683b6a408d7b0274ba6 Mon Sep 17 00:00:00 2001 From: regisss Date: Wed, 30 Nov 2022 21:15:33 +0100 Subject: [PATCH 2/4] Make style --- docs/source/optimization/habana.mdx | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/docs/source/optimization/habana.mdx b/docs/source/optimization/habana.mdx index 64fd16a7ff49..775f1ec8a3d5 100644 --- a/docs/source/optimization/habana.mdx +++ b/docs/source/optimization/habana.mdx @@ -33,6 +33,7 @@ Finally, you will need to specify a [Gaudi configuration](https://huggingface.co ```python from optimum.habana import GaudiConfig from optimum.habana.diffusers import GaudiDDIMScheduler, GaudiStableDiffusionPipeline + model_name = "stabilityai/stable-diffusion-2-base" scheduler = GaudiDDIMScheduler.from_pretrained(model_name, subfolder="scheduler") pipeline = GaudiStableDiffusionPipeline.from_pretrained( @@ -47,7 +48,10 @@ pipeline = GaudiStableDiffusionPipeline.from_pretrained( You can then call the pipeline to generate images by batches from one or several prompts: ```python outputs = pipeline( - prompt=["High quality photo of an astronaut riding a horse in space", "Face of a yellow cat, high resolution, sitting on a park bench"], + prompt=[ + "High quality photo of an astronaut riding a horse in space", + "Face of a yellow cat, high resolution, sitting on a park bench", + ], num_images_per_prompt=10, batch_size=4, ) From ff4fd140a41ebed6ea92ae55f0e58bfcb4e942c4 Mon Sep 17 00:00:00 2001 From: regisss Date: Wed, 30 Nov 2022 22:26:28 +0100 Subject: [PATCH 3/4] Add benchmark --- docs/source/optimization/habana.mdx | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/docs/source/optimization/habana.mdx b/docs/source/optimization/habana.mdx index 775f1ec8a3d5..b6d4821704d2 100644 --- a/docs/source/optimization/habana.mdx +++ b/docs/source/optimization/habana.mdx @@ -28,7 +28,7 @@ To generate images with Stable Diffusion 1 and 2 on Gaudi, you need to instantia When initializing the pipeline, you have to specify `use_habana=True` to deploy it on HPUs. Furthermore, in order to get the fastest possible generations you should enable **HPU graphs** with `use_hpu_graphs=True`. -Finally, you will need to specify a [Gaudi configuration](https://huggingface.co/docs/optimum/habana/package_reference/gaudi_config) which can be downloaded from the H[ugging Face Hub](https://huggingface.co/Habana). +Finally, you will need to specify a [Gaudi configuration](https://huggingface.co/docs/optimum/habana/package_reference/gaudi_config) which can be downloaded from the [Hugging Face Hub](https://huggingface.co/Habana). ```python from optimum.habana import GaudiConfig @@ -58,3 +58,13 @@ outputs = pipeline( ``` For more information, check out Optimum Habana's [documentation](https://huggingface.co/docs/optimum/habana/usage_guides/stable_diffusion) and the [example](https://github.com/huggingface/optimum-habana/tree/main/examples/stable-diffusion) provided in the official Github repository. + + +## Benchmark + +Here are the latencies for Habana Gaudi 1 and Gaudi 2 with the [Habana/stable-diffusion](https://huggingface.co/Habana/stable-diffusion) Gaudi configuration (mixed precision bf16/fp32): + +| | Latency | Batch size | +| ------- | ------- | ---------- | +| Gaudi 1 | 4.37s | 4/8 | +| Gaudi 2 | 1.19s | 4/8 | From 5ac391ff3b78c57330adc36f479909f6d490fcb0 Mon Sep 17 00:00:00 2001 From: regisss Date: Wed, 30 Nov 2022 22:49:00 +0100 Subject: [PATCH 4/4] Center-align columns in the benchmark table --- docs/source/optimization/habana.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/optimization/habana.mdx b/docs/source/optimization/habana.mdx index b6d4821704d2..064eb510ce24 100644 --- a/docs/source/optimization/habana.mdx +++ b/docs/source/optimization/habana.mdx @@ -65,6 +65,6 @@ For more information, check out Optimum Habana's [documentation](https://hugging Here are the latencies for Habana Gaudi 1 and Gaudi 2 with the [Habana/stable-diffusion](https://huggingface.co/Habana/stable-diffusion) Gaudi configuration (mixed precision bf16/fp32): | | Latency | Batch size | -| ------- | ------- | ---------- | +| ------- |:-------:|:----------:| | Gaudi 1 | 4.37s | 4/8 | | Gaudi 2 | 1.19s | 4/8 |