v1.10.0: Stable Diffusion XL pipelines

Stable Diffusion XL

Enable SD XL ONNX export and ONNX Runtime inference by @echarlaix in #1168

• Enable SD XL ONNX export using the CLI :

optimum-cli export onnx --model stabilityai/stable-diffusion-xl-base-0.9 --task stable-diffusion-xl ./sd_xl_onnx

• Add SD XL pipelines for ONNX Runtime inference (supported tasks : text-to-image and image-to-image) :

from optimum.onnxruntime import ORTStableDiffusionXLPipeline

model_id = "stabilityai/stable-diffusion-xl-base-0.9"
pipeline = ORTStableDiffusionXLPipeline.from_pretrained(model_id, export=True)

prompt = "sailing ship in storm by Leonardo da Vinci"
image = pipeline(prompt).images[0]
pipeline.save_pretrained("onnx-sd-xl-base-0.9")

Stable Diffusion pipelines

Enable image-to-image and inpainting pipelines for ONNX Runtime inference by @echarlaix in #1121

More examples in documentation

Major bugfixes

• Fix bloom KV cache usage in ORTForCausalLM by @fxmarty in #1152

What's Changed

• Add stable diffusion example by @prathikr in #1136
• Fixed incomplete ONNX export model memory release issue by @sharpbai in #1154
• Add trust remote code option for config by @changwangss in #1151
• Fix typos of ONNXRuntimme -> ONNXRuntime by @mgoin in #1155
• Fix ONNX export for MobileViT for segmentation by @regisss in #1128
• Revert "update the default block size" by @rui-ren in #1162
• ONNX export for custom architectures & models with custom modeling code by @fxmarty in #1166
• Update Optimum Neuron doc by @regisss in #1164
• Fix stable diffusion ONNX export by @echarlaix in #1173
• Add gpt_bigcode model_type to NormalizedTextConfig by @changwangss in #1170
• Allow attention_mask=None for BetterTransformer in the inference batched case for gpt2 & gpt-neo by @fxmarty in #1180
• Fix encoder attention mask input order for ORT by @fxmarty in #1181
• Fix ORTModel initialization on specific device id by @fxmarty in #1182
• Add stable diffusion img2img and inpain documentation by @echarlaix in #1149
• Fix SD XL ONNX export for img2img task by @echarlaix in #1194
• Remove graphcore from documentation quickstart by @echarlaix in #1201
• Unpin tensorflow by @fxmarty in #1211
• Fix ORT test for unknown architecture for task by @fxmarty in #1212
• add ort + stable diffusion documentation by @prathikr in #1205
• Fix vision encoder decoder that may not cache cross-attention by @fxmarty in #1210
• Add documentation for Optimum Furiosa by @regisss in #1165
• Add BLIP-2 to BetterTransformer documentation by @fxmarty in #1218
• Set default value to unet config sample size by @echarlaix in #1223
• Fix broken link in doc by @regisss in #1222
• Fix BT test by @fxmarty in #1224
• Add SD XL documentation by @echarlaix in #1198
• Update setup.py to add optimum-furiosa extras by @mht-sharma in #1226

New Contributors

• @sharpbai made their first contribution in #1154
• @mgoin made their first contribution in #1155

Full Changelog: v1.9.0...v1.10.0