zentorch Release v5.2.1

ZenDNN Plugin for PyTorch (zentorch) v5.2.1 Release Notes

Overview

Incremental release built on the zentorch 5.2.0 foundation. Adds PyTorch 2.11.0 support, with vLLM compatibility (0.15.0–0.18.0), and strengthens asymmetric WOQ and INT8 dynamic quantization capabilities.

Key Improvements

Framework Support

• Support for PyTorch 2.11.0 (in addition to 2.10.0).
• Extended vLLM compatibility: 0.16.0, 0.17.0, 0.17.1, and 0.18.0.

Performance & Quantization Enhancements

• Optimized vLLM RMSNorm using C++ CPU kernels.
• Asymmetric WOQ support via TorchAO:
  • Int4 weight-only quantization (Int4WeightOnlyOpaqueTensorConfig).
  • Bias support for asymmetric quantized ops.
  • Int4OpaqueTensor support using zentorch.
  • Fusion support for quantized linear operators.
• INT8 dynamic quantization:
  • Integrated dynamic qlinear operator in zentorch backend.
  • Enabled execution of dynamically quantized models via vLLM with zentorch.
• Improved WOQ linear operator performance.

Infrastructure, Testing & Tooling

• Expanded Hypothesis-based testing for qlinear and BMM operations.
• Accuracy benchmarking using the LM Eval framework.
• New zentorch weekly PyPI package for development builds.
• Recommended jemalloc via LD_PRELOAD for DLRMv2 quantized model execution.

Bug Fixes & Accuracy

• Fixed unit tests for qlinear_mul_add, fused WOQ linear, and BMM hypothesis tests.

Compatibility & Known Issues

• Validated only with torchao==0.16.0 for quantized models.
• No breaking changes; fully backward compatible with zentorch 5.2.0.
• Known issues:
  • Possible GLIBCXX conflicts (may require LD_PRELOAD).
  • Experimental Python versions (3.13T, 3.14, 3.14T) are not supported.