You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
* clipping for fp16
* fix typo
* added fp16 inference to docs
* fix docs typo
* include link for fp16 investigation
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Flux can generate high-quality images with FP16 (i.e. to accelerate inference on Turing/Volta GPUs) but produces different outputs compared to FP32/BF16. The issue is that some activations in the text encoders have to be clipped when running in FP16, which affects the overall image. Forcing text encoders to run with FP32 inference thus removes this output difference. See [here](https://github.com/huggingface/diffusers/pull/9097#issuecomment-2272292516) for details.
82
+
83
+
FP16 inference code:
84
+
```python
85
+
import torch
86
+
from diffusers import FluxPipeline
87
+
88
+
pipe = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-schnell", torch_dtype=torch.bfloat16) # can replace schnell with dev
89
+
# to run on low vram GPUs (i.e. between 4 and 32 GB VRAM)
90
+
pipe.enable_sequential_cpu_offload()
91
+
pipe.vae.enable_slicing()
92
+
pipe.vae.enable_tiling()
93
+
94
+
pipe.to(torch.float16) # casting here instead of in the pipeline constructor because doing so in the constructor loads all models into CPU memory at once
95
+
96
+
prompt ="A cat holding a sign that says hello world"
97
+
out = pipe(
98
+
prompt=prompt,
99
+
guidance_scale=0.,
100
+
height=768,
101
+
width=1360,
102
+
num_inference_steps=4,
103
+
max_sequence_length=256,
104
+
).images[0]
105
+
out.save("image.png")
106
+
```
107
+
80
108
## Single File Loading for the `FluxTransformer2DModel`
81
109
82
110
The `FluxTransformer2DModel` supports loading checkpoints in the original format shipped by Black Forest Labs. This is also useful when trying to load finetunes or quantized versions of the models that have been published by the community.
0 commit comments