Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
88 changes: 88 additions & 0 deletions auto_round/compressors/diffusion/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
# AutoRound for Diffusion Models (Experimental)

This feature is experimental and may be subject to changes, including potential bug fixes, API modifications, or adjustments to default parameters.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

link this file to step_by_step.md

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done


## Quantization

### API Usage (CPU/GPU) Recommended

By default, AutoRoundDiffusion only quantizes the transformer module of diffusion models and uses `COCO2014 captions` for calibration.

```python
import torch
from auto_round import AutoRound
from diffusers import AutoPipelineForText2Image

# Load the model
model_name = "black-forest-labs/FLUX.1-dev"
pipe = AutoPipelineForText2Image.from_pretrained(model_name, torch_dtype=torch.bfloat16)

# Quantize the model
autoround = AutoRound(
pipe,
scheme="MXFP8",
dataset="coco2014",
num_inference_steps=10,
guidance_scale=7.5,
generator_seed=None,
batch_size=1,
)
autoround.quantize()

# Save the quantized model
output_dir = "./tmp_autoround"
# Currently loading the quantized diffusion model is not supported, so use fake format
autoround.save_quantized(output_dir, format="fake", inplace=True)
```

- `dataset`: the dataset for quantization training. Currently only support coco2014 and user customized .tsv file.

- `num_inference_steps`: The reference number of denoising steps.

- `guidance_scale`: Control how much the image generation process follows the text prompt. The more it is, the more closely it follows the prompt.

- `generator_seed`: A seed that controls the initial noise from which an image is generated.

for more hyperparameters introduction, please refer [Homepage Detailed Hyperparameters](../../README.md#api-usage-gaudi2cpugpu)

### CLI Usage

A user guide detailing the full list of supported arguments is provided by calling ```auto-round -h``` on the
terminal.

```bash
auto-round \
--model black-forest-labs/FLUX.1-dev \
--scheme MXFP8 \
--format fake \
--batch_size 1 \
--output_dir ./tmp_autoround
```

### Diffusion Support Matrix

For diffusion models, currently we only validate quantizaion on the FLUX.1-dev, which involves quantizing the transformer component of the pipeline.

| Model | calibration dataset |
|--------------|--------------|
| black-forest-labs/FLUX.1-dev | COCO2014 |



<details>
<summary style="font-size:17px;">Calibration Dataset</summary>

For diffusion models, we used [**coco2014**]("https://github.com/mlcommons/inference/raw/refs/heads/master/text_to_image/coco2014/captions/captions_source.tsv") calibration dataset as our default.

If users want to use their own dataset, please build the dataset file in ".tsv" format following below structure and use it through argument --dataset (tsv file):
```
id caption
0 YOUR_PROMPT
1 YOUR_PROMPT
... ...
```
- `id`: The id used to map generated images and prompts.
- `caption`: The text prompt used to generate the images.


</details>
2 changes: 1 addition & 1 deletion docs/step_by_step.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
Step-by-Step
============

This document presents step-by-step instructions for auto-round llm quantization. For vlms quantization, please refer to [vlms user guide](../auto_round/mllm/README.md)
This document presents step-by-step instructions for auto-round llm quantization. You can refer to [vlms user guide](../auto_round/compressors/mllm/README.md) for vlms quantization and [diffusions user guide](../auto_round/compressors/diffusion/README.md) for diffusions quantization.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@n1ck-guo please refine the mllm doc following the comments in this pr


* [1 Prerequisite](#1-prerequisite)
* [2 Prepare Calibration Dataset](#2-prepare-calibration-dataset)
Expand Down