Fix Cohere compatibility logit scaling - #1727
Conversation
| """ | ||
| return state_dict | ||
|
|
||
| def postprocess_weights(self, bridge: Any) -> None: |
There was a problem hiding this comment.
supported_architectures/AGENTS.md "When to override preprocess_weights" tells adapter authors to fold any config knob HF applies in forward, with Cohere's logit_scale as its worked example and a copy-paste skeleton. Following it produces exactly this bug, because it never says the forward-time factor also has to be taken out.
Please add postprocess_weights to that section: when the factor lives outside the module the bridge wraps, folding it is only half the change.
There was a problem hiding this comment.
Addressed in 1544c4ce. I expanded the preprocess_weights guidance in supported_architectures/AGENTS.md to cover the paired post-install lifecycle: when HF applies a factor outside the wrapped component, adapters now track whether the fold occurred and use postprocess_weights() to neutralize the outer runtime factor after installation. The guidance and skeleton also call out that a missing runtime attribute means there is no outer factor to neutralize.
| return | ||
|
|
||
| model = getattr(bridge, "original_model", None) | ||
| if model is None or not hasattr(model, "logit_scale"): |
There was a problem hiding this comment.
If original_model has no logit_scale, its forward isn't applying one, so the fold alone is already correct and there is nothing to neutralize. A Cohere-architecture module in that shape processes cleanly on dev and hard-fails here. This occurs after the folded weights are installed and _weights_processed is set, so a caller that catches the error is left holding the mis-scaled bridge.
Return instead of raising when the attribute is absent.
There was a problem hiding this comment.
Addressed in 1544c4ce. postprocess_weights() now neutralizes model.logit_scale only when the wrapped model exposes it, and always clears the pending fold state without raising when the attribute is absent. I added a network-free regression that removes the model-level attribute, runs process_weights(), and verifies that processing completes, the attribute remains absent, the unembed weight is folded by 0.5, and cfg.logit_scale remains 0.5. Focused validation passed: 3 end-to-end regressions, 74 Cohere/Cohere2 adapter tests, 27 weight-processing tests with 1 skip, formatting, and mypy over 392 source files.
Description
Fixes #1726
Cohere and Cohere2 weight processing folds
cfg.logit_scaleinto the live unembedding weights, but the Hugging Face model forward still applied its model-level scale afterward. As a result, the first valid compatibility-mode conversion produced logits with the scale applied twice.This change adds an architecture post-install lifecycle hook. The Cohere adapter uses it to neutralize the Hugging Face model-level
logit_scaleonly after the scaled unembedding weights have been installed. The Bridge config retains the original scale as metadata, and tied input embeddings remain unscaled.Network-free tiny-config regressions cover both direct
process_weights()and the firstenable_compatibility_mode()call.Type of change
Validation
tests/integration/model_bridge/test_cohere_adapter.py: 24 passedmypy .: Success, no issues found in 392 source filesgit diff --check: passedChecklist: