Skip to content

Release v0.13.0

Latest
Latest

Choose a tag to compare

View all tags
@Giuseppe5 Giuseppe5 released this 14 Jul 16:36
v0.13.0
f6c8e2e

Highlights

The most important changes in this release cycle:

  • (MX)FP quantizers moved out of experimental — floating-point / MX quantizers are now first-class, supported quantizers. (#1488)
  • Experimental vLLM export support — new export path targeting vLLM. (#1444)
  • Learned Round refactor + documentation — reworked learned round in the common examples with dedicated docs. (#1323, #1485)
  • LLM customization plugins — custom trainer support and a custom quantizer plugin to modify the quantized model, plus the ability to pass custom quantizers via the common quantizer API. (#1474, #1508, #1456)
  • GGUF improvements — quantize high-impact layers to high precision and attach metadata to GGUF exports. (#1532, #1527)
  • PeRQ (formerly MixQuant) — initial MixQuant support added, then renamed to PeRQ throughout the codebase and docs. (#1448, #1521, #1525)
  • Python 3.13 support. (#1451)
  • Utility to replace weights with quantized ones. (#1505)
  • Signed scaling — signed scaling stats in core and signed scales in the LLM entry point. (#1393, #1449)
  • Blockwise / block rotations — support for blockwise rotations and the option to disable fused block rotations. (#1434, #1438)
  • New search algorithms for LLM experiments (#1357)

Breaking Changes

  • LLM: load as much data as needed — data loading behavior changed in the LLM examples. (#1514)
  • Removed Python 3.9 support. (#1481)
  • Deprecated old Python and Torch versions. (#1392)
  • numpy is now an optional extra — installs no longer pull in numpy by default. (#1403)
  • Minimum transformers version now enforced. (#1510)
  • CLI rename: block_rotation_dim → rotation_block_size. (#1464)
  • Deprecated unused MLIR export in the LLM examples. (#1467)

All Commits

Features

  • Feat (benchmark): Allow easy specification of different search algorithms within benchmark scripts (#1357)
  • Feat (brevitas_examples/llm): custom trainer support (#1474)
  • Feat (ex/axe): improved implementation with extended support/testing (#1181)
  • Feat (gguf): enabling quantization of high-impact layers to high precision (#1532)
  • Feat (utils): utils to replace weights with quantized ones (#1505)
  • Feat (utils): reducing variance on power_iteration func (#1519)
  • Feat (ex/llm): custom quantizer plugin allows to modify quant model (#1508)
  • Feat (quant/float): move (MX)FP quantizers out of experimental (#1488)
  • Feat (brevitas_examples/llm)!: load as much data as needed (#1514)
  • Feat (llm): enabling narrow_range for inputs in LLM entry point (#1524)
  • Feat (benchmark): Minor tweaks to run_args_bucket_process (#1523)
  • Feat (loss): support for high precision loss (#1496)
  • Feat (docs): Documentation for Learned Round (#1485)
  • Feat (vLLM): experimental support for vLLM export (#1444)
  • Feat (ex/common): learned round refactor (#1323)
  • Feat (brevitas_examples/llm): better dataset handling (#1461)
  • Feat (brevitas_examples/llm): Use dataloader in AWQ (#1489)
  • Feat (ex/llm): Signed scales in LLM entrypoint (#1449)
  • Feat (brevitas_examples/llm): deprecate unused MLIR export (#1467)
  • Feat (cli): rename block_rotation_dim to rotation_block_size (#1464)
  • Feat (brevitas_examples/llm): fineweb support (#1459)
  • Feat (bit_width): generalize restrict_impl with clamping (#1457)
  • Feat (equalize): adding initial support for MixQuant (#1448)
  • Feat (common/quantizer): Enable passing custom quantizers (#1456)
  • Feat (brevitas_examples/llm): compute perplexity at float32 (#1458)
  • Feat (equalize): option to disable fused block rotations (#1438)
  • Feat (optim): decoupling dtype of CayleySGD from params (#1443)
  • Feat (brevitas_examples/llm): Support for batched inputs in GPXQ/Qronos (#1427)
  • Feat (brevitas_examples/llm): better RMSNorm replacement (#1436)
  • Feat (brevitas_examples/common): mse scale for weights with float quant (#1433)
  • Feat (brevitas_examples/llm): support for blockwise rotations (#1434)
  • Feat (graph/equalize): minor refactor (#1432)
  • Feat (core/stats): signed scaling stats (#1393)
  • Feat (core): module for runtime computation of exp bias (#1418)
  • Feat (brevitas_examples/llm): support for distillation loss (#1388)
  • Feat (quant/mx): Added midmax scale rounding option to MX types (#1409)
  • Feat (nn): add HardSwish activation function (#1406)
  • Feat (gpxq): enabling selection of device and dtype (#1405)
  • Feat (loss/bit_width): removable hooks (#1407)
  • Feat (export/onnx): fallback export to fake quantized weights (#1395)
  • Feat (core/float): better max mantissa computation (#1391)
  • Feat (quant/float): configurable float bit-width implementations (#1373)
  • Feat (brevitas_examples/llm): adding test split (#1375)

Fixes

  • Fix (stats): Mix MSE for zero-point (#1491)
  • Fix (fx): push support for value_tracer (#1538)
  • Fix (tests/permute): use symbolic_trace instead of dynamo (#1539)
  • Fix (tests/permute): use symbolic_trace instead of dynamo (#1536)
  • Fix (export/onnx): Update ONNX dynamo export for PT2.9 API (#1526)
  • Fix (core/float): better handling of continuous mantissa bit-width (#1528)
  • Fix (brevitas_examples/llm): patch dynamo FX with torch 2.10+ (#1533)
  • Fix (core/stats): typo in variable name in MeanLearnedSigmaStd (#1530)
  • Fix (brevitas_examples/gguf): adding metadata to GGUF export (#1527)
  • Fix (brevitas_examples/llm): correct zero-shot handling of thinking models (#1518)
  • normalization fix and new tests (#1504)
  • Fix: restore model_cache_implementation to model.generation_config (#1516)
  • Fix (stats): Set correct reduce_dim for group_dim<0 (#1512)
  • Fix (brevitas_examples/data): fix evaluation of wikitext2 (#1495)
  • Fix (ex/llm): Allow setting dataset limit for lighteval (#1492)
  • Fix (graph/gpxq): ConvTranspose temporarily unsupported (#1490)
  • Fix (brevitas_examples/llm): enable wikitest test split with fineweb and pile (#1487)
  • Fix (brevitas_examples/llm): remove_hooks after fused_rotation_no_fx (#1480)
  • Fix (graph/equalize): general check for fast_hadamard_transform (#1484)
  • Fix (fx): bump version guard for value tracer (#1482)
  • Fix (rotation): fix extra compile keys in state dict (#1472)
  • Fix (brevitas/quant): Fix MX MSE quantizer (#1468)
  • Fix (brevitas_examples/llm): fix to RMSNorm context manager (#1471)
  • Fix (graph): enable ROCm path in _apply_had_device (#1469)
  • Fix (brevitas_examples/llm): args based BOS for lm_eval (#1462)
  • Fix (brevitas_examples/llm): compile support with merged rotations (#1429)
  • Fix (graph/qronos): Normalize contribution to H and G when buffer is disabled (#1440)
  • Fix (brevitas_examples/llm): fix transformers tests (#1446)
  • Fix (brevitas_examples/quantizers): correct stats for dynamic quants (#1445)
  • Fix (brevitas_examples/llm): more checks for FX-related args (#1441)
  • Fix (brevitas_examples/llm): correct batch size for lm_eval (#1430)
  • Fix (proxy): preserve training state after tensor_quant re-init (#1419)
  • Fix (core): import scaling before zero point to prevent circular import (#1422)
  • Fix (src/brevitas/quant_tensor/base_quant_tensor.py): Added torch.float64 to dict of tolerances (#1417)
  • Fix (brevitas_examples/imagenet/ptq): DataLoader Fix (#1420)
  • Fix (equalize): Fix LayerwiseActivationRotation (#1413)
  • Fix (graph/rotation): small refactor and documentation (#1382)
  • Fix (nn/bias): propagate runtime_shape from QuantScaleBias (#1385)
  • Fix (core/scaling): fix dtype for int threshold (#1404)
  • Fix (Notebooks): Corrected typos/small errors in the text cells with explanations (#1399)
  • Fix (export/onnx): import GLOBALS from correct location depending on torch version (#1398)
  • Fix (ex/llm): Fix integration with Lighteval Python API (#1379)
  • Fix (ex/llm): Recursively unwrap equalized layer (#1390)
  • Fix (paper/expand): Avoid skipping baseline experiment (#1383)

Setup & Dependencies

  • Setup: minimum version for transformers (#1510)
  • Setup: support for python 3.13 (#1451)
  • Deps (python): remove python 3.9 support (#1481)
  • Setup: unpin transformers (#1415)
  • Deps (ex/llm): Fixed perf<0.18 to be compatible with transformers==4.50.0 (#1421)
  • Deps (numpy): switched numpy to be an optional extra (#1403)
  • Deps (onnxscript): limit onnxscript<0.5.4 (#1400)
  • Setup: deprecate old python and torch versions (#1392)
  • Setup: pin numexpr version (#1389)
  • Setup: fix requirements for HF packages (#1386)
  • Deps: comment out GH dep

Docs

  • Docs: update dev docs (#1531)
  • Docs: update PeRQ and Qronos references in docs (#1525)
  • Docs: renaming MixQuant to PeRQ (#1521)
  • Docs: documentation for MixQuant (#1470)
  • Docs: correct index in Papers section (#1476)
  • Docs: post-training expansion documentation (#1465)
  • Docs: generate dev and v0.12.1 docs

Tests & CI

  • Test (ex/llm): remove redundant mock from lighteval tests (#1493)
  • Test (qonnx): Fixes to QONNX export tests when (dynamo=True) (#1473)
  • Test: test ONNX dynamo export with torch >= 2.7 (#1455)
  • Test (brevitas_examples/llm): new lm_eval tests (#1431)
  • Test (brevitas_examples/llm): added test of lighteval with rotation (#1425)
  • Test (graph): restore skipped tests (#1374)
  • Test (diffusers): skip torch 2.1 due to diffusers incompatibility (#1378)
  • CI (periodic): Added periodic tests for the entire suite of supported versions (#1368)

New Contributors

Full Changelog: v0.12.1...v0.13.0

Contributors

surajkarki66 and xkucerak
Assets 2
Loading