Skip to content

v0.16.0 Release

Choose a tag to compare

View all tags
@wenhuach21 wenhuach21 released this 30 Sep 12:36
· 57 commits to main since this release
v0.16.0
This tag was signed with the committer’s verified signature.
XuehaoSun Sun, Xuehao
GPG key ID: 9A95988F4E45920A
Verified
Learn about vigilant mode.
9b6214f
This commit was signed with the committer’s verified signature.
XuehaoSun Sun, Xuehao
GPG key ID: 9A95988F4E45920A
Verified
Learn about vigilant mode.

Highlights

  • [Performance] Speedup 2.5-6X moe tuning and pin ngram to cpu by @wenhuach21 in #2303
  • [Performance] Stream AutoScheme's per-layer sensitivity scoring block-by-block by @aquilarubra in #2063
  • [Backend]support 5,6,7 woq bits on cuda in Transformers and vLLM by @wenhuach21 in #2281
  • [Performance] implement resume functionality for model-free compression by @xin3he in #2330
  • [Accuracy]fix: restore FP32 tensors saved as FP16/BF16 during tensor copying by @xin3he in #2375
  • [Calibration]add fineweb-edu dataset as calibration backup by @WeiweiZhang1 in #2345
  • [Document]update awq detail data of fake eval by @WeiweiZhang1 in #2285
  • [ARK] ARK updates #2193 #2280 , etc
  • [Diffusion] Diffusion updates #2251 #2314, etc

What's Changed

  • Add more test cases to improve CUDA UT coverage by @lvliang-intel in #2159
  • Simplify default calibration dataset log by @changwangss in #2162
  • fix DiffusionCompressionOrchestrator missing nsamples by @changwangss in #2179
  • Enable Torch compilation for throughput tests by @xin3he in #2181
  • fix: support aggregated Granite MoE tensors in gguf export by @n1ck-guo in #2175
  • adjust lrx2 logic to fix its issue in mixed bits scenarios and add more woq support in llmc by @wenhuach21 in #2164
  • [Experimental] Add NVFP4 E5M3 support and related components for model-free quantization by @xin3he in #2123
  • Fix wan i2v pipeline by @changwangss in #2180
  • For compatibility with Stable Diffusion by @changwangss in #2191
  • update readme by @wenhuach21 in #2194
  • Increase memory limits for the build process in the CI pipeline by @chensuyue in #2198
  • Reduce SYCL-TLA Compiler Memory by @luoyu-intel in #2193
  • feat: add support for FP8 model loading and ragged-tile dequantization fallback by @xin3he in #2158
  • fix: layer_config for lm_head takes effect in model-free mode by @xin3he with @Copilot in #2182
  • Fix CT export in mixed‑precision mode by @1607662794 in #2128
  • add workaround for llmc issue and add unit tests for model_free_utils helpers by @xin3he in #2207
  • refactor: optimize expert selection and output handling in moe forward by @xin3he in #2209
  • Preserve diffusion component dtypes during quantization by @changwangss in #2205
  • disable timeout in pytest until found a more stable method by @chensuyue in #2222
  • add awq torch_compile fix, disable_opt_rtn by default for awq by @WeiweiZhang1 in #2208
  • update auto_scheme_acc doc by @n1ck-guo in #2000
  • Update alg combination news by @wenhuach21 in #2227
  • add enable_torch_compile mark to SGLang and vLLM throughput tests by @xin3he in #2226
  • add rotation ut for xpu. by @lkk12014402 in #2094
  • Fix FP8 quantization config for native fp8 export by @Zhenzhong1 in #2228
  • fix: AutoScheme parallel scoring OOMs (MoE memory, VLM gate) and disk-stream tuning crash by @avtc in #2220
  • refactor uts, remove duplicate uts and build common uts for all devices to increase coverage by @n1ck-guo in #2157
  • Fix Cuda UT coverage collect by @chensuyue in #2234
  • Fix CUDA UT issue by @xin3he in #2238
  • test: skip FP8 tests with transformers 5.16.x by @changwangss in #2241
  • increase precision of loss logging in quantization tuning for debugging by @xin3he in #2229
  • test: skip HPU FP8 test with transformers 5.16.x by @changwangss in #2247
  • Update troubleshooting tips for quantization issues in README_CN.md by @XuehaoSun in #2244
  • fix: remove stale quantizer access in dual-transformer diffusion by @changwangss in #2236
  • fix: preserve diffusion mixed dtypes during loading by @changwangss in #2232
  • fix argument mismatch by @xin3he in #2248
  • [Bugfix] Fix Sglang e2e UT issue by @lvliang-intel in #2250
  • disable torch compile in rtn/opt-rtn/iters<10 and fix glm-5.3 tuning exception by @wenhuach21 in #2246
  • bump release version by @chensuyue in #2257
  • fix: warm up Triton CUDA driver before mocking device capability by @XuehaoSun in #2253
  • Bug fix: Replace WAWrapperLinear with FakeActQuantLinear instead of Linear by @xin3he in #2245
  • feat: Add support for deterministic algorithms flags in CLI and update logging behavior by @xin3he in #2261
  • fix: kimi 2.6 dequantization is not correct by @xin3he in #2260
  • improve "compressed-tensors" import error raise by @changwangss in #2263
  • Update CI workflow Infra by @chensuyue in #2255
  • Enhance CI coverage reporting with hardware-specific configurations by @XuehaoSun in #2259
  • support custom max shard size in CLI and model export by @n1ck-guo in #2258
  • refactor algorithm cli by @n1ck-guo in #2140
  • Update RunPod management to the V2 API. by @XuehaoSun in #2271
  • fix(diffusion): use real images for I2V calibration by @changwangss in #2272
  • fix: asymmetric zero-point packing, format-scoped 8-bit asym, W5/W6/W7 llm_compressor presets by @avtc in #2273
  • fix: replace bare except clauses with except Exception by @ramkrishs in #1981
  • fix: skip activation fake-quant in FakeActQuantLinear when act quantization is not configured by @n1ck-guo in #2286
  • feat: Enhance test logging and retry mechanism for unit tests by @chensuyue in #2274
  • feat(orchestrator): add reference_output to quantize_block to skip collect_reference by @xesdiny in #2289
  • fix CUDA UT by @xin3he in #2284
  • Fix: cpu sdpa fp16-bf16 repack route unmatched test by @jijiaz in #2292
  • fix gguf format mtp bug by @n1ck-guo in #2297
  • Allow non-MLLM calibration datasets when not quantizing vision by @aquilarubra in #2065
  • update awq detail data of fake eval by @WeiweiZhang1 in #2285
  • Optimize INT4 MoE prefill/decode (sym) XPU kernels with dedicated w4a16 tiling by @a32543254 with @Copilot in #2111
  • [risky]reduce ram usage of loading moe without monkey patch and fix qwen3-flash-next group_size issue by @wenhuach21 in #2291
  • Update CI/CD dockerfile by @XuehaoSun in #2294
  • Fix xpu CI retry issue by @chensuyue in #2309
  • support 5,6,7 bits on cuda and add humming backend by @wenhuach21 in #2281
  • fix: handle mantissa round overflow by @xin3he in #2302
  • add sdxl svdquant by @changwangss in #2251
  • Optimize diffusion calibration step sampling by @changwangss in #2265
  • fix: update quantization configuration handling in FakeFormat and related tests by @xin3he in #2311
  • fix: defer packing for multi-transformer diffusion by @changwangss in #2314
  • support model_type == "inkling_mm_model" by @xin3he in #2214
  • reduce cuda ut runing time by @n1ck-guo in #2266
  • Implement AI-Powered CI Failure Analysis and Fix Recommendations by @chensuyue in #2277
  • fix: update artifact name format for ark wheel download by @chensuyue in #2321
  • [high risk]Speedup 2.5-6X moe tuning and pin ngram to cpu by @wenhuach21 in #2303
  • fix: pin Hugging Face snapshots for AutoScheme offload reloads by @n1ck-guo in #2318
  • fix nemoron gate inference issue by @wenhuach21 in #2325
  • refactor: reorganize tensor utility functions and enhance split_fused_expert_tensors for special cases by @xin3he in #2320
  • feat: add support for WOQ fake quantization in model-free mode and update… by @xin3he in #2329
  • feat: add transformers checkpoint conversion mapping support for missing tensors by @xin3he in #2328
  • XPU oneDNN: bind matmul execution to live SYCL queue (graph-capture safe) by @luoyu-intel with @Copilot in #2280
  • Improve the low_cpu_mem_usage usage in the doc by @hshen14 in #2278
  • fix: preserve layer config for secondary diffusion transformers by @changwangss in #2337
  • fix: recreate calibrator for secondary diffusion transformers by @changwangss in #2335
  • feat: add opt-in diffusion tuning cache prefetch by @changwangss in #2340
  • Keep the original checkpoint key names and fix Qwen4 vLLM inference by @wenhuach21 in #2327
  • Update README.md by @wenhuach21 in #2343
  • feat: implement resume functionality for model-free compression by @xin3he in #2330
  • fix qwen35 regression by @wenhuach21 in #2352
  • fix: reload Diffusers blocks from component checkpoints by @changwangss in #2353
  • fix: update key cache access for compatibility with newer DynamicCache implementation by @chensuyue in #2348
  • fix: support list inputs in diffusion tuning cache by @changwangss in #2346
  • fix: preserve declared FP32 tensors during model dtype conversion by @changwangss in #2359
  • Refactor test timeouts and update dependencies for performance by @XuehaoSun in #2361
  • unify target_bits/options into bits/schemes for AutoScheme by @n1ck-guo in #2333
  • perf: avoid unused SVD reconstruction and select the CUDA driver in SVDQuant by @changwangss in #2356
  • Add HMT+Quant(mxfp4) kernel to Ark by @jijiaz in #2334
  • fix: save the config of diffusers models on export by @Ar4ikov in #2119
  • refactor: decouple datatype quantization from algorithms by @n1ck-guo in #2360
  • fix: recognize Modular Diffusers pipelines as diffusion models by @Ar4ikov in #2120
  • add blockwise rotation. by @lkk12014402 in #2087
  • docs: clarify diffusion export format by @changwangss in #2367
  • feat: add NVFP4 static KV cache quantization support by @yiliu30 in #2366
  • add fineweb-edu dataset as calibration bakeup by @WeiweiZhang1 in #2345
  • remove bagel specific constraints by @WeiweiZhang1 in #2372
  • bug fix: fix NVFP4 evaluation with input_ global_scale handling and testing by @xin3he in #2374
  • Remove deprecated files by @chensuyue in #2373
  • patch triangular_solve for Qwen3.8 model for hpu by @lkk12014402 in #2377
  • [ARK] Improve XPU W4A16 WOQ decode with dense S4 DPAS path by @Zhenzhong1 in #2331
  • fix: share NVFP4 global scales for Wan QKV projections by @changwangss in #2379
  • fix: restore FP32 tensors saved as FP16/BF16 during tensor copying by @xin3he in #2375
  • fix:reduce memory usage when loading native fused MoE checkpoints by @n1ck-guo in #2382
  • fix: update import to include get_model_path for model retrieval by @xin3he in #2388
  • fix: fake format has no FP32 tensor restoration by @xin3he in #2392
  • fix model_free saving with single shard and MXFP fake format evaluation by @xin3he in #2391
  • Cast float32 weights to the export dtype when saving by @Shubham-Padkonde in #2385
  • Apply autocast only when mixed precision is enabled by @Shubham-Padkonde in #2393
  • Move the zero point to CPU for asymmetric layers too by @Shubham-Padkonde in #2394
  • Honour the padding mask in SignRoundV2's plain loss by @Shubham-Padkonde in #2395
  • feat: use OpenS2V for diffusion calibration by @changwangss in #2368
  • ark: add BF16/FP16 Sparge variants and reduce compilation RAM by @yiliu30 in #2145
  • fix: update tensor summary creation to directly use tensors_to_restor… by @xin3he in #2401
  • fix: update get_predefined_ignore_layers to include shared_expert_gat… by @xin3he in #2397
  • Revert "refactor: decouple datatype quantization from algorithms by @n1ck-guo in #2406
  • fix: avoid excessive memory during calibration dataset casting by @changwangss in #2405
  • fix: materialize checkpoint-backed cpu buffers by @n1ck-guo in #2412
  • Fix CWE-22 (path traversal) in checkpoint loading issues by @lvliang-intel in #2389
  • bug fix: layer_config mismatch due to transformers rename; fix model free stream mode detection by @xin3he in #2411
  • enhance attention module with original implementation retrieval… by @xin3he in #2408
  • bug fix: prune static kv/attention granularity keys when dtype not set by @yiliu30 in #2415
  • fix(xpu): make ARK SDPA graph-capture safe for torch.xpu.XPUGraph by @luoyu-intel in #2416

New Contributors

Full Changelog: v0.15.1...v0.16.0

Contributors

ramkrishs, avtc, and 20 other contributors
Assets 2
Loading