Skip to content

v1.0.3: docs: Changelog 1.0.3 (SDXL SageAttention2, release notes link)

Choose a tag to compare

View all tags
@ussoewwin ussoewwin released this 05 Feb 12:10
· 1786 commits to main since this release
v1.0.3
96be888

SDXL HSWQ SageAttention2 Integration (v1.0.3)

Overview

SDXL HSWQ V1.2 and V1.6 integrate SageAttention2 to speed up calibration. Enable it with the --sa2 option. The same FP8 output is produced; SA2 is used only during the calibration loop.

Architecture

  1. Detect --sa2 flag — If set, attempt to import SageAttention2.
  2. Import and enable — try_import_sage_attention() then enable_sage_attention() monkey-patches torch.nn.functional.scaled_dot_product_attention.
  3. Run calibration loop — Forward passes use the patched SDPA (SA2) where applicable.
  4. Remove hooks and disable SA2 — After calibration, hooks are removed and disable_sage_attention() restores the original SDPA.
  5. Compute quantization parameters — Amax and FP16-keep list are computed as usual (no SA2 involved).

Implementation (code-accurate)

Global state: _sage_attn_available (import success), _original_sdpa (original SDPA reference).

Import: try_import_sage_attention() does from sageattention import sageattn and sets _sage_attn_available = True on success.

Enable: Replaces F.scaled_dot_product_attention with a wrapper that:

  • If attn_mask is not None or is_causal is true: calls original SDPA with the same arguments (SA2 does not support these).
  • Otherwise: calls sageattn(query, key, value, is_causal=False). On any exception, falls back to original SDPA with the same arguments.

Disable: Restores F.scaled_dot_product_attention = _original_sdpa and clears _original_sdpa.

Fallback behavior

Condition Behavior
attn_mask provided Use original SDPA
is_causal=True Use original SDPA
SA2 runtime error Use original SDPA
SA2 not installed Warning and standard SDPA for calibration

Usage

# V1.2 (standard precision)
python quantize_sdxl_hswq_v1.2.py --input model.safetensors --output model_fp8.safetensors --calib_file prompts.txt --sa2

# V1.6 (high precision)
python quantize_sdxl_hswq_v1.6.py --input model.safetensors --output model_fp8.safetensors --calib_file prompts.txt --sa2

Why SA2 is safe during calibration

  1. Statistics only — Calibration collects weight amax and sensitivity statistics. Small numerical differences from SA2 do not affect the result.
  2. Inference output discarded — Generated images are not used; only statistics from intermediate activations are used.
  3. Amax uses normal path — SA2 is active only during the calibration forward passes; quantization parameter (amax) computation runs at full precision.

SA2 vs FlashAttention2

Item SageAttention2 FlashAttention2
Speed ~10% faster Baseline
Precision INT8 quantized (small error) Full precision
Windows ✅ (e.g. woct0rdho fork) ❌ (community build needed)
Ease of use pip install Build required

Script versions

Script SA2 (--sa2) Quantization
V1.1 (archives) ❌ Standard (bins=4096)
V1.2 ✅ Standard (bins=4096, 200 candidates, 3 refinements)
V1.5 (archives) ❌ High precision (bins=8192, 1000 candidates, 10 refinements)
V1.6 ✅ High precision (bins=8192, 1000 candidates, 10 refinements)

Dependencies

SageAttention2 (optional). Example for Windows:

pip install https://github.com/woct0rdho/SageAttention/releases/download/v2.2.0-windows.post4/sageattention-2.2.0+cu130torch2.9.0andhigher.post4-cp39-abi3-win_amd64.whl

Check the SageAttention repository for your platform and PyTorch/CUDA version.

Assets 2
Loading