Skip to content

v0.72.1 — Streamed adapters are normal adapters

Choose a tag to compare

@MakazhanAlpamys MakazhanAlpamys released this 27 Jul 18:39
· 566 commits to main since this release

If you trained with stream_layers: true on v0.72.0, that adapter is inert — re-run or re-save it on v0.72.1.

What's New

This is an out-of-band correctness release. It fixes one defect in v0.72.0's layer streaming and ships nothing else.

Layer-streaming adapters were saved in an unloadable form. The streaming wrapper holds the real decoder layer as a child named inner, so every saved LoRA tensor carried an extra .inner. segment in its key:

base_model.model.model.layers.0.inner.self_attn.q_proj.lora_A.weight
                               ^^^^^^

soup merge, soup serve, soup chat and PeftModel.from_pretrained all loaded zero adapter tensors from such a file and silently returned the untuned base model. PEFT emitted only a UserWarning, so nothing failed and nothing looked wrong.

The training itself was correct — the streamed run's numerics are unaffected and v0.72.0's bit-exactness results still stand. Only the saved file was affected.

Check an existing adapter

python -c "from safetensors.torch import load_file; \
print([k for k in load_file('adapter_model.safetensors') if '.inner.' in k][:3])"

If that prints anything, the adapter is affected. Re-running is the reliable fix. From v0.72.1 a streamed adapter is byte-for-byte in the same layout as an ordinary LoRA run, and portable to any tool that has never heard of layer streaming.

Also fixed

  • --hf-resume bypassed the streaming resume refusal. The guard tested only --resume, while --hf-resume reaches resume_from through a different branch. That combination previously appeared to work by accident; once adapters are saved canonically it would instead have matched nothing and continued with a freshly initialised adapter, silently. Both flags are now refused for streaming runs.
  • Roadmap renumbered, since this release was inserted ahead of NF4. Every "this lands in vX.Y.Z" refusal message was corrected: NF4 streaming is v0.72.2; the disk tier, more architectures, larger batches, gradient accumulation and checkpoint/resume are v0.72.3; preference losses are v0.72.4.

Install / Upgrade

pip install --upgrade "soup-cli[train]"

Security

No security-relevant changes. No schema change, no new dependency, no change to training behaviour.

Known Limitations

  1. Existing v0.72.0 adapters are not repaired by upgrading — the file on disk still carries .inner. keys. Detection is the one-liner above; re-run or re-save on v0.72.1. A key-rewriting repair command was deliberately not shipped: it would be a new user-facing surface in a hotfix, and a mis-mapped key produces a subtly wrong model rather than an obvious failure.
  2. Loading into a streaming run stays unsupported. The fix is serialisation-only by design, so in memory state_dict() and named_parameters() still disagree. --resume / --hf-resume are refused for streaming runs, naming v0.72.3.

Found by the v0.72.2 NF4 gate, not by v0.72.0's own test suite — none of its 159 streaming tests saved an adapter and loaded it back. The new regression test does exactly that: by count, by name and by value, with a negative control that re-mangles the keys and asserts the reload yields zeros, because 0-of-N loading raises no exception.