Skip to content

Comments

Quantization: Add 2bit to precision constants, Fix -1 group_size handling in ModelBuilder#2338

Merged
jambayk merged 2 commits intomainfrom
2bit-mb
Feb 18, 2026
Merged

Quantization: Add 2bit to precision constants, Fix -1 group_size handling in ModelBuilder#2338
jambayk merged 2 commits intomainfrom
2bit-mb

Conversation

@jambayk
Copy link
Contributor

@jambayk jambayk commented Feb 17, 2026

Describe your changes

Checklist before requesting a review

  • Add unit tests for this change.
  • Make sure all tests can pass.
  • Update documents if necessary.
  • Lint and apply fixes to your code by running lintrunner -a
  • Is this a user-facing change? If yes, give a description of this change to be included in the release notes.

(Optional) Issue link

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds support for 2-bit quantization and fixes handling of per-channel quantization (group_size == -1) in the ModelBuilder pass. The changes enable proper loading and processing of quantized models that use per-channel quantization or 2-bit precision.

Changes:

  • Added BITS2 = 2 to the PrecisionBits enum to support 2-bit quantization
  • Fixed ModelBuilder logic to handle group_size == -1 (per-channel quantization) correctly by avoiding division by -1
  • Enabled and enhanced test coverage for ModelBuilder with Olive-quantized models, including group_size == -1 test cases

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File Description
olive/constants.py Added BITS2 constant to PrecisionBits enum for 2-bit quantization support
olive/passes/onnx/model_builder.py Refactored qweight tensor processing to properly handle group_size == -1 and simplified the logic by removing separate scales handling
test/passes/onnx/test_model_builder.py Removed skip decorator and added group_size parametrization to test both normal (16) and per-channel (-1) quantization

@jambayk jambayk merged commit 86f9469 into main Feb 18, 2026
17 checks passed
@jambayk jambayk deleted the 2bit-mb branch February 18, 2026 20:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants