Skip to content

Commit a169571

Browse files
sayakpaulecharlaix
andauthored
[docs] Update ONNX doc to use optimum (#2702)
* minor edits to onnx and openvino docs. * Apply suggestions from code review Co-authored-by: Ella Charlaix <80481427+echarlaix@users.noreply.github.com> --------- Co-authored-by: Ella Charlaix <80481427+echarlaix@users.noreply.github.com>
1 parent f4bbcb2 commit a169571

File tree

2 files changed

+31
-39
lines changed

2 files changed

+31
-39
lines changed

docs/source/en/optimization/onnx.mdx

Lines changed: 30 additions & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -13,61 +13,53 @@ specific language governing permissions and limitations under the License.
1313

1414
# How to use the ONNX Runtime for inference
1515

16-
🤗 Diffusers provides a Stable Diffusion pipeline compatible with the ONNX Runtime. This allows you to run Stable Diffusion on any hardware that supports ONNX (including CPUs), and where an accelerated version of PyTorch is not available.
16+
🤗 [Optimum](https://github.com/huggingface/optimum) provides a Stable Diffusion pipeline compatible with ONNX Runtime.
1717

1818
## Installation
1919

20-
- TODO
20+
Install 🤗 Optimum with the following command for ONNX Runtime support:
21+
22+
```
23+
pip install optimum["onnxruntime"]
24+
```
2125

2226
## Stable Diffusion Inference
2327

24-
The snippet below demonstrates how to use the ONNX runtime. You need to use `OnnxStableDiffusionPipeline` instead of `StableDiffusionPipeline`. You also need to download the weights from the `onnx` branch of the repository, and indicate the runtime provider you want to use.
28+
To load an ONNX model and run inference with the ONNX Runtime, you need to replace [`StableDiffusionPipeline`] with `ORTStableDiffusionPipeline`. In case you want to load
29+
a PyTorch model and convert it to the ONNX format on-the-fly, you can set `export=True`.
2530

2631
```python
27-
# make sure you're logged in with `huggingface-cli login`
28-
from diffusers import OnnxStableDiffusionPipeline
29-
30-
pipe = OnnxStableDiffusionPipeline.from_pretrained(
31-
"runwayml/stable-diffusion-v1-5",
32-
revision="onnx",
33-
provider="CUDAExecutionProvider",
34-
)
32+
from optimum.onnxruntime import ORTStableDiffusionPipeline
3533

34+
model_id = "runwayml/stable-diffusion-v1-5"
35+
pipe = ORTStableDiffusionPipeline.from_pretrained(model_id, export=True)
3636
prompt = "a photo of an astronaut riding a horse on mars"
37-
image = pipe(prompt).images[0]
37+
images = pipe(prompt).images[0]
38+
pipe.save_pretrained("./onnx-stable-diffusion-v1-5")
3839
```
3940

40-
The snippet below demonstrates how to use the ONNX runtime with the Stable Diffusion upscaling pipeline.
41+
If you want to export the pipeline in the ONNX format offline and later use it for inference,
42+
you can use the [`optimum-cli export`](https://huggingface.co/docs/optimum/main/en/exporters/onnx/usage_guides/export_a_model#exporting-a-model-to-onnx-using-the-cli) command:
4143

42-
```python
43-
from diffusers import OnnxStableDiffusionPipeline, OnnxStableDiffusionUpscalePipeline
44+
```bash
45+
optimum-cli export onnx --model runwayml/stable-diffusion-v1-5 sd_v15_onnx/
46+
```
47+
48+
Then perform inference:
49+
50+
```python
51+
from optimum.onnxruntime import ORTStableDiffusionPipeline
4452

53+
model_id = "sd_v15_onnx"
54+
pipe = ORTStableDiffusionPipeline.from_pretrained(model_id)
4555
prompt = "a photo of an astronaut riding a horse on mars"
46-
steps = 50
47-
48-
txt2img = OnnxStableDiffusionPipeline.from_pretrained(
49-
"runwayml/stable-diffusion-v1-5",
50-
revision="onnx",
51-
provider="CUDAExecutionProvider",
52-
)
53-
small_image = txt2img(
54-
prompt,
55-
num_inference_steps=steps,
56-
).images[0]
57-
58-
generator = torch.manual_seed(0)
59-
upscale = OnnxStableDiffusionUpscalePipeline.from_pretrained(
60-
"ssube/stable-diffusion-x4-upscaler-onnx",
61-
provider="CUDAExecutionProvider",
62-
)
63-
large_image = upscale(
64-
prompt,
65-
small_image,
66-
generator=generator,
67-
num_inference_steps=steps,
68-
).images[0]
56+
images = pipe(prompt).images[0]
6957
```
7058

59+
Notice that we didn't have to specify `export=True` above.
60+
61+
You can find more examples in [optimum documentation](https://huggingface.co/docs/optimum/).
62+
7163
## Known Issues
7264

7365
- Generating multiple prompts in a batch seems to take too much memory. While we look into it, you may need to iterate instead of batching.

docs/source/en/optimization/open_vino.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,4 +36,4 @@ prompt = "a photo of an astronaut riding a horse on mars"
3636
images = pipe(prompt).images[0]
3737
```
3838

39-
You can find more examples in [optimum documentation](https://huggingface.co/docs/optimum/intel/inference#export-and-inference-of-stable-diffusion-models).
39+
You can find more examples (such as static reshaping and model compilation) in [optimum documentation](https://huggingface.co/docs/optimum/intel/inference#export-and-inference-of-stable-diffusion-models).

0 commit comments

Comments
 (0)