v0.74.0 — the base was loaded in fp32 the whole time

116 of the 120 merged pull requests in this release came from outside the maintainer, by 25 people. The maintainer's own four were #484, #486, #498 and #509.

What's New

The frozen base was being loaded in fp32 the whole time (#339 by @blackcoderx in #471). All three from_pretrained sites — text, vision, audio — passed no explicit dtype, so a base that never receives an optimizer step was materialised at twice its checkpoint precision. Measured on an H100 with Llama-3.1-8B + LoRA: 48,241 MiB -> 18,658 MiB peak, 2.59x / 28.9 GB, byte-identical across three repeats. A trainable base still loads fp32, deliberately and documented. The full-fine-tune discriminator is now one shared is_full_finetune() used by both the trainer and the VRAM pre-flight — previously two independent copies that disagreed in both directions.

Transformers 5.x, TRL 0.29, PEFT 0.20 (#502/#503 by @Amix29 in #507). Qwen3.5-family text decoders train on the Transformers path, TRL APIs that moved under trl.experimental are reached by capability probe rather than a version table, and pip install "soup-cli[train,mlx]" resolves again — the two extras previously declared ranges that could not be satisfied together at all.

The free Colab/Kaggle tier could not stream (#425/#429). T4 / P100 / V100 / GTX 16xx crashed stream_layers: true with _amp_foreach_non_finite_check_and_unscale_cuda not implemented for 'BFloat16', because peft creates LoRA adapters in the base checkpoint's dtype while the fp16 GradScaler requires fp32 gradients. Trainable *lora_* params are cast to fp32 before optimizer creation; the full-FT paths are deliberately untouched so trainable memory does not double after the pre-flight.

soup train --cloud lambda (#264 by @kok-o in #528). Plan-only by default. The API key never enters the instance: termination belongs to a local controller, and the finally both terminates and polls to confirm it happened.

Also: soup mcp serve on both mcp majors with the <2 cap lifted, chosen by probing the Server constructor rather than reading mcp.__version__ (#322 in #498) · unknown config keys warned with a "did you mean" suggestion and a stated v0.75 rejection deadline (#627 by @Srinivasan8888 in #628) · per-PR changelog fragments replacing the shared [Unreleased] section (#487 by @Amix29 in #490) · LISA accepting task: pretrain (#307 by @ousamabenyounes in #476) · the DeepSpeed empty-LoRA-group guard extended from one wrapper to all eighteen that needed it (#359 in #484) · fourteen new recipes, catalog now 163.

Install / Upgrade

pip install -U soup-cli
pip install -U "soup-cli[train]"

Security

Four bypasses of the same shape, closed on both validators (#600/#599 by @here-2007 in #604/#624, #616 in #625): abbreviated (127.1), decimal (2130706433), hex (0x7f000001) and octal (0177.0.0.1) IPv4 spellings reached the telemetry and webhook guard — and, through a path the first fix never touched, the OTLP tracing endpoint validator.

Breaking: /v1/tools/bash is re-enabled under real OS-level namespace/sandbox isolation (#151 by @kok-o in #527), and soup serve now exits 2 when bound to a non-loopback host without --tool-auth-token, where it previously only printed a warning. The endpoint it protects now actually executes code, so a warning was no longer a sufficient control.

The SGLang backend also now obeys the --trust-remote-code gate instead of loading every model with it enabled (#360 by @Srinivasan8888 in #619). Breaking by design: a custom-code model on that backend now fails to load without the flag.

Known Limitations

• The declared torch>=2.5.0 floor does not work with trl>=0.29. Measured, not inferred: at torch 2.5.1 torch.distributed.fsdp.FSDPModule does not exist, so trl 0.29.1 cannot import and DPO / KTO / GRPO / BCO are unavailable. pip cannot catch it because trl declares no torch dependency at all, and CI never sees it because >=2.5.0 always resolves to the newest torch. A fresh install is unaffected; a pinned 2.5.x environment is not. Tracked as #651. No floor bump ships here because 2.5.1 was measured to fail and 2.6 was not measured to work.
• soup doctor never reports MLX, and get_mlx_version() always returns "unknown" — #659, fix in review.
• Layer streaming remains BETA.
• The preprint's measured numbers are unchanged, but its scope is now narrower than the code: it states nine streaming architectures and the code admits ten. It stays explicitly scoped to v0.73.0.

Contributors

Every pull request below came from outside the maintainer.

• @Amix29 (34) — the Transformers 5.x / TRL 0.29 / PEFT 0.20 migration and the [train,mlx] resolution conflict; per-PR changelog fragments, which removes the shared-file merge conflict without making changelog loss silent; Apple Silicon monitoring; SmolVLM/Idefics3 vision batches; the two-phase Best-of-N workflow and its durability work.
• @Srinivasan8888 (14) — the unknown-config-key warning with a stated rejection deadline, then returning to cover the one uncovered line of his own guard; the SGLang trust_remote_code gate, which he deliberately kept out of an earlier PR so it could be argued on its own terms; lisa_train_embeddings; four recipes.
• @AmirF194 (13) — live-eval quantization threading; Cut-CE architecture resolution; branch coverage in pure-Python helpers, including one that corrected the issue's own premise.
• @umran666 (7) — reconciling the two reward-hack ladders by establishing what the code actually does rather than adding a mechanism; the log_level foreign-handler branch; the kimi-k2.6-grpo recipe.
• @harshitthek (6) — MLX device detection, where the label was never the harm: every Apple Silicon run silently rewrote quantization: 4bit to none; GEMM repeat selection; path-separator normalisation.
• @ousamabenyounes (4) — LISA extended to continued pre-training; configurable layer-streaming pinning; the physical-RAM ceiling for the RAM tier.
• @blackcoderx (4) — the fp32 upcast, this release's headline; data.interleave wired into training and then extended to streaming and hub-name lists.
• @AchuthReddy-16 (4) — torch_dtype at every production load site plus the CI cell that exercises the transformers floor; mapping-like tokenizer outputs; deferred callback resolution.
• @SID-6921 (4) — the recipe-config snapshot guard and its delta encoding; MLX --resume, which had been silently restarting from scratch.
• @kok-o (3) — the Lambda Cloud backend, whose third revision removed the need for the credential to travel rather than patching the leak; /v1/tools/bash under OS-level isolation.
• @here-2007 (3) — the obfuscated-IPv4 SSRF work, including the tiered guard.
• @Samearth17 (3) — the required PyTorch smoke job, and the published bit-exactness and bitsandbytes reproduction harnesses.
• @ARAVIND281 (3) — the MII backend's chat template and real finish_reason; ORPO/SimPO streaming coverage; the VRAM fit measured on a second GPU and software stack.
• @Faisal01011 (2) — FSDP + QLoRA dtype compatibility; provider sampling in Best-of-N.
• @Nick-800 (2) — eleven recipes across two PRs (Qwen2.5-Coder/Math and the R1-Distill families).
• @CODING-DARSH, @Shutaru, @lesterppo, @Osheun, @taljeon, @k4its1t, @chadetov, @MKnaomi2, @webdevsamran, @YuriPerro (1 each) — cross-tokenizer draft support; Qwen3.5 MoE streaming; the pre-Ampere GradScaler crash; the glm-5.1-dpo recipe; concrete Transformers model-load coverage; extended dtype-guard coverage; the SFT chat-template fixture; a CUDA wheel recommendation the driver can actually run; the peak-GPU figure in the live panel; and the batch probe gated on the measured peak rather than only on OutOfMemoryError.

Full detail, with every entry credited inline: CHANGELOG.md.