v0.72.1 — Streamed adapters are normal adapters
If you trained with
stream_layers: trueon v0.72.0, that adapter is inert — re-run or re-save it on v0.72.1.
What's New
This is an out-of-band correctness release. It fixes one defect in v0.72.0's layer streaming and ships nothing else.
Layer-streaming adapters were saved in an unloadable form. The streaming wrapper holds the real decoder layer as a child named inner, so every saved LoRA tensor carried an extra .inner. segment in its key:
base_model.model.model.layers.0.inner.self_attn.q_proj.lora_A.weight
^^^^^^
soup merge, soup serve, soup chat and PeftModel.from_pretrained all loaded zero adapter tensors from such a file and silently returned the untuned base model. PEFT emitted only a UserWarning, so nothing failed and nothing looked wrong.
The training itself was correct — the streamed run's numerics are unaffected and v0.72.0's bit-exactness results still stand. Only the saved file was affected.
Check an existing adapter
python -c "from safetensors.torch import load_file; \
print([k for k in load_file('adapter_model.safetensors') if '.inner.' in k][:3])"If that prints anything, the adapter is affected. Re-running is the reliable fix. From v0.72.1 a streamed adapter is byte-for-byte in the same layout as an ordinary LoRA run, and portable to any tool that has never heard of layer streaming.
Also fixed
--hf-resumebypassed the streaming resume refusal. The guard tested only--resume, while--hf-resumereachesresume_fromthrough a different branch. That combination previously appeared to work by accident; once adapters are saved canonically it would instead have matched nothing and continued with a freshly initialised adapter, silently. Both flags are now refused for streaming runs.- Roadmap renumbered, since this release was inserted ahead of NF4. Every "this lands in vX.Y.Z" refusal message was corrected: NF4 streaming is v0.72.2; the disk tier, more architectures, larger batches, gradient accumulation and checkpoint/resume are v0.72.3; preference losses are v0.72.4.
Install / Upgrade
pip install --upgrade "soup-cli[train]"Security
No security-relevant changes. No schema change, no new dependency, no change to training behaviour.
Known Limitations
- Existing v0.72.0 adapters are not repaired by upgrading — the file on disk still carries
.inner.keys. Detection is the one-liner above; re-run or re-save on v0.72.1. A key-rewriting repair command was deliberately not shipped: it would be a new user-facing surface in a hotfix, and a mis-mapped key produces a subtly wrong model rather than an obvious failure. - Loading into a streaming run stays unsupported. The fix is serialisation-only by design, so in memory
state_dict()andnamed_parameters()still disagree.--resume/--hf-resumeare refused for streaming runs, naming v0.72.3.
Found by the v0.72.2 NF4 gate, not by v0.72.0's own test suite — none of its 159 streaming tests saved an adapter and loaded it back. The new regression test does exactly that: by count, by name and by value, with a negative control that re-mangles the keys and asserts the reload yields zeros, because 0-of-N loading raises no exception.