feat: FP8 quantization & TensorRT build infrastructure

[forkni]

Summary

• Adds complete FP8 quantization pipeline via nvidia-modelopt: ONNX export → FP16 optimize → FP8 Q/DQ annotation → TRT STRONGLY_TYPED engine
• Adds fp8=True parameter flow from StreamDiffusionWrapper.__init__() through _load_model(), compile_unet(), and EngineBuilder.build()
• Engine path gains --fp8 suffix for separate cache (e.g. ...--controlnet--fp8--mode-img2img/unet.engine)
• Two-pass cleanup in builder.py prevents ~14 GB intermediate file bloat on Windows (ONNX weights + onnx__* tensor files)
• Fixes 4 TRT 10.12 compatibility bugs in FP8 build path (STRONGLY_TYPED, version-aware precision flags)
• Patches ByteSize() for >2GB ONNX models in modelopt calibration (required for SDXL UNet)

Stacking

Stacks on pr3/ipadapter-vram-deps (which stacks on pr1/inference-performance). PR3 provides onnx/onnxruntime/modelopt dependency pins that FP8 imports require.

Commits (8)

1. 61bcf86 — patch ByteSize() for >2GB ONNX in modelopt FP8 quantization
2. 7a02ae2 — reduce FP8 calibration batches 128→8 (KVO cache OOM)
3. 402d619 — merge calibration list-of-dicts into stacked dict for modelopt
4. 7c05f59 — add NVIDIA DLLs to PATH and retry without quantize_mha on ORT EP failure
5. 5ba15af — use single calibration batch, cleanup intermediates on retry
6. 6b0b99d — resolve 4 FP8 bugs for TRT 10.12 (STRONGLY_TYPED network, version-aware flags)
7. 24da142 — prevent intermediate file bloat on Windows (two-pass cleanup, onnx__* early deletion)
8. 670aec4 — add FP8 parameter flow: wrapper → compile_unet → engine path prefix

Files Modified

File	Changes
acceleration/tensorrt/fp8_quantize.py	New file (464 lines): calibration data generation, quantize_onnx_fp8()
acceleration/tensorrt/builder.py	FP8 build stage, two-pass cleanup, build stats ordering fix
acceleration/tensorrt/utilities.py	_build_fp8() raw TRT builder, onnx__* early cleanup, external data support
acceleration/tensorrt/models/models.py	get_dynamic_axes() output axes for FP8 compatibility
acceleration/tensorrt/__init__.py	compile_unet() extracts fp8/calibration_data_fn from engine_build_options
acceleration/tensorrt/engine_manager.py	--fp8 suffix in UNet engine path
wrapper.py	fp8= param, self.fp8, passthrough to _load_model() and get_engine_path()

Test plan

• FP16 baseline engine builds without --fp8 suffix (no regression)
• StreamDiffusionWrapper(fp8=True) generates engine path with --fp8 suffix
• FP8 ONNX (*.fp8.onnx) preserved after build; intermediates cleaned
• Engine directory contains only unet.engine, unet.fp8.onnx, build_stats.json
• FP16 fallback triggers if modelopt not installed (graceful degradation)

🤖 Generated with Claude Code