Skip to content

Fix transformers 5.2.0 transpose conversion issue#1475

Merged
chensuyue merged 4 commits intomainfrom
lvl/fix_transpose_conversion_issue
Mar 10, 2026
Merged

Fix transformers 5.2.0 transpose conversion issue#1475
chensuyue merged 4 commits intomainfrom
lvl/fix_transpose_conversion_issue

Conversation

@lvliang-intel
Copy link
Contributor

Description

Root cause
In transformers 5.2.0, the new Transpose conversion class calls get_parameter() on weight tensors during quantized MoE checkpoint loading. auto_round's QuantLinear stores weight_packed and weight_scale as buffers (via register_buffer()), not nn.Parameter, so get_parameter() raises an AttributeError, which transformers then raises as a RuntimeError.

Fix
A _patch_transpose_for_buffers() monkey-patch wraps Transpose.convert() to check if the target tensor is a buffer first — if so, it returns the tensor as-is instead of attempting a transpose. This patch is applied in monkey_patch_transformers() for transformers ≥ 5.2.0.

Type of Change

  • Bug fix
  • New feature
  • Documentation update
  • Performance improvement
  • Code refactoring
  • Other (please specify):

Related Issues

Fixes or relates to #

Checklist Before Submitting

  • My code has been tested locally.
  • Documentation has been updated as needed.
  • New or updated tests are included where applicable.

Signed-off-by: lvliang-intel <liang1.lv@intel.com>
Copilot AI review requested due to automatic review settings February 27, 2026 07:33
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a compatibility monkey-patch to auto_round to prevent failures when loading quantized MoE checkpoints with transformers==5.2.0+, where a new transpose conversion path incorrectly assumes weights are nn.Parameter rather than buffers.

Changes:

  • Introduce _patch_transpose_for_buffers() to wrap transformers.core_model_loading.Transpose.convert() and bypass transpose logic for buffer-backed tensors.
  • Apply the patch from monkey_patch_transformers() for transformers >= 5.2.0.

lvliang-intel and others added 2 commits February 27, 2026 15:39
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@chensuyue chensuyue merged commit 0127590 into main Mar 10, 2026
29 checks passed
@chensuyue chensuyue deleted the lvl/fix_transpose_conversion_issue branch March 10, 2026 03:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants