Skip to content

Conversation

@LRL2-ModelCloud
Copy link
Collaborator

No description provided.

@Qubitium
Copy link
Collaborator

Qubitium commented Oct 24, 2025

@avtc Watch when this PR gets merged. I still need fix some details. This fixes a HF Transformers bug where if you save the model is MISSING a mtp layer for GLM Moe (4.5, 4.6 Air included) which is optional but used by vllm/sglang to speed up inference!

@Qubitium Qubitium changed the title copy mtp file and update index file Fix GLM 4.5/4.6 and AIr not saving mtp layer after save (HF bug) Oct 24, 2025
@Qubitium Qubitium merged commit 1a5ebb8 into main Oct 24, 2025
4 checks passed
@Qubitium Qubitium deleted the save-mtp-safetensors branch October 24, 2025 16:31
avtc added a commit to avtc/GPTQModel that referenced this pull request Oct 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants