v0.73.3 — every pull request came from outside

Every one of the 24 pull requests in this release came from someone other than the maintainer — from eight people, five of whom appear here for the first time. The maintainer's own work in this window arrived as four direct commits (license headers, a test repair, a CI guard), not as pull requests.

What they found is the more interesting number: four separate config flags that were validated by the schema, documented, and then read by nothing.

What's New

Four flags that did nothing

• training.bnb_4bit_use_double_quant was read by nothing. Every 4-bit construction site hardcoded use_double_quant=True, so setting it to false changed your config fingerprint and nothing else. The fix's design choice is the interesting half: the field is Optional[bool] = None, not a plain True, because a True default emits the key into model_dump() and trips the footgun guard on re-validation — measured, 21 of 173 shipped configs stopped round-tripping, which would have broken train --replay and soup sweep. (#321)
• On Apple Silicon, quantization: 4bit was silently rewritten to none. detect_device() did not know MLX, so every run reported "CPU (no GPU detected)" and quietly downgraded. The label was never the harm. The quantization decision is now an explicit, testable resolve_quantization() rather than a branch buried in a 900-line function that has 0% coverage. (#423)
• soup train --no-reexec printed a launch command with your own flags missing. Follow it literally and you trained without --fsdp — and the run succeeded, so nothing pointed back at the hint. Two hand-maintained copies of "what the user typed"; the printed one is deleted and the hint now derives from the argv that actually launches the run. (#372)
• data.interleave validates, documents, parses — and nothing reads it at training time. Found during review of the mix-recipe fix and filed rather than papered over. (#443)

Silent data corruption in assistant-only masking

A tokenizer returning BatchEncoding — which is not a dict — slipped past the type guard, so the label mask was built from the mapping's key strings: no exception, no warning, a normal-looking loss curve, and zero trained tokens. (#430)

Windows: a process that exited with code 259 read as alive forever

259 is also STILL_ACTIVE, so the two were indistinguishable. It defeated run reconciliation and could wedge the MCP execution cap shut with no error an operator could act on. Disambiguated with WaitForSingleObject, and the two ~50-line copies of the liveness check collapsed into one shared module with a test asserting identity, so a third copy fails. (#424)

New: soup mcp serve --allow-execute

Runs a planned training or export behind a short-lived, single-use, server-generated confirmation token bound to the plan and the execution kind — no command, no argv, no shell string, no client-supplied environment. shell=False, stdin=DEVNULL, output to .soup/mcp-runs/<run_id>.log, and the run goes through the existing tracker so soup runs sees what MCP started. (#297)

Also

soup ship's leg-1 noise floor now measured in the judge task modes, not only metric (#403) · eval.ship.noise_floor committable to soup.yaml (#406) · orphaned running MCP runs reconciled on read, because the watcher is a daemon thread killed without unwinding (#401) · the one-active-execution cap gated on a live persisted run, so a restarted server cannot double-book a GPU (#402) · detect_disk_kind seeing through virtio, where a paravirtual device reports rotational: 1 and a measured 1.5 GB/s cloud disk was refused as an HDD (#365) · MitigationLogWriter no longer drops every record when its parent directory vanishes mid-run (#343) · soup draft distill --steps N delivers ~N steps instead of ~N/4.44 (#364) · \boxed {A} with a space before the brace no longer scores as no-answer (#396) · soup env check audits installed versions against Soup's own declared bounds (#368) · bom and attestation as first-class registry artifact kinds so soup card can link them (#309) · two new recipes, qwen3.5-4b-pretrain and deepseek-v4-flash-grpo (#278, #279).

Install / Upgrade

pip install --upgrade "soup-cli[train]"
soup version

Python 3.10–3.12. No config migration is needed and no default changes: bnb_4bit_use_double_quant left unset still resolves to the historical True, so no existing 4-bit run changes behaviour.

Security

• MCP gated execution (#297) closes a real gap between planning and running. The config is snapshotted at plan time and executed from the copy, so the original cannot be edited underneath the run; and digest_file now walks a directory tree by content (sorted relative paths + per-file hash, symlink refusal, bounded) rather than by mtime+size, which did not change when a file inside a protected directory was rewritten — a model could previously be swapped between plan and execution and revalidation still passed. Token consumption and capacity acquisition happen before Popen, so a failed spawn requires a fresh plan rather than enabling replay. --allow-execute is a separate, stronger opt-in than --allow-mutating.
• Windows process liveness (#424) — the 259 collision could wedge the execution cap shut indefinitely. Fixed, and the platform split is deliberate: os.kill(pid, 0) is a genuine existence check on POSIX but on Windows CPython routes signal 0 to GenerateConsoleCtrlEvent, i.e. it sends a console Ctrl+C.

Known Limitations

• #394 (MLX) remains open. The PR in this release hardens the dispatch route so backend: mlx cannot reach the PyTorch/TRL stack; it does not fix the reported hang, and its author retitled it to say so.
• #371 (reward-hack controller) remains open. #414 addresses one fix-path item and is marked Refs, not Fixes.
• #442 — soup data mix --live writes an overlay config that fails to load, the same defect as #330 in a second renderer. Invisible because every --live test mocks subprocess.run, so the artifact is built and never loaded.
• #443 — data.interleave has no training-time reader. This is why the mix recipe collapses to a single dataset rather than emitting the mixture it searched for; it needs a deliberate decision, not another workaround.
• #444 — a CUDA-gated GEMM-ceiling test is flaky under machine load. CI never sees it (it skips without a GPU), which is exactly why it is filed rather than left to surprise someone.

No gate record accompanies this release: nothing here was gated by a new measurement, so benchmarks/ is unchanged. The preprint is likewise untouched — no measured number moves and its scope is unchanged.

Contributors

24 of the 24 pull requests in this release came from outside the maintainer.

Ben Younes (@ousamabenyounes) — eleven merges

• MitigationLogWriter dropped every record in silence once its parent directory vanished mid-run, so the controller kept acting while its evidence stopped growing (#398)
• soup draft distill --steps N delivered only ~N/4.44 optimiser steps — val_split and gradient_accumulation_steps both divide the budget and the epoch arithmetic ignored both (#399)
• The leg-2 MCQ scorer read \boxed {A} as no-answer: LaTeX permits a space before the brace, models emit it, and the cue tier cannot rescue it (#396)
• --noise-floor was the one soup ship gate-policy flag with no config surface, so a team enforcing a floor in CI could not commit it (#410)
• A dead MCP watcher left its run at running forever, because the watcher is a daemon thread killed without unwinding (#407)
• The one-active-execution cap lived in process memory, so a restarted server saw a free slot and could double-book a training already running (#408)
• The leg-1 noise floor was measured in --task-mode metric only, so in the judge modes a win smaller than the instrument's own resolution still counted as a win (#419)
• detect_disk_kind could not see through virtio — a paravirtual device reports rotational: 1 — so a measured 1.5 GB/s cloud disk was classified HDD and refused the streaming disk tier (#411)
• training.bnb_4bit_use_double_quant was validated and then read by nothing: every 4-bit path hardcoded True, so setting it changed the config fingerprint and nothing else (#418)
• soup env check now audits the live environment against the bounds Soup declares about itself, catching pip install vllm quietly pushing transformers past the <5.0.0 cap (#421)
• bom and attestation were not registry artifact kinds, so the two compliance documents soup card exists to publish were the two it could not link (#420)

Darsh (@CODING-DARSH) — eighth merge

• Gated train_execute / export_execute behind a single-use, server-generated confirmation token — no command, no argv, no client environment — with the config snapshotted at plan time and protected directories digested by content rather than by mtime, so a model swapped between plan and execution no longer revalidates (#393)
• Corrected a contributor's handle carried in the previous release's CHANGELOG — a one-line fix to somebody else's credit, which is the kind of thing that normally goes unmade (#400)

Amir Fathi (@AmirF194) — fourth merge

• kl_control re-wrote the same β on every hold step, so a run that never tripped the controller was numerically identical to log_only rather than the no-op it claims to be; the mitigation log now carries held / acted / released as a field instead of free text (#414)

Faisal Fayaz (@Faisal01011) — first four merges

• Added the qwen3.5-4b-pretrain recipe with a test pinning the literal repo id — the only check that catches an id wrong consistently in both RecipeMeta.model and the inline base:, the defect class that once shipped glm-5 pointing at the wrong org (#422)
• Added the deepseek-v4-flash-grpo recipe, carrying that same literal-id guard a second time (#432)
• materialize_meta_adapters() returns a count, and on newer PEFT it returns 0 as a matter of course — so 0 stopped distinguishing "nothing to do" from a silent no-training run; the decision moved into a real postcondition the caller cannot ignore (#435)
• Then took the non-blocking follow-up from that review unprompted: the guard's lora_ restriction was called deliberate in its own docstring and nothing pinned it, so a later broadening would have begun refusing healthy streamed builds (#437)

Achuth Reddy Bangaru (@AchuthReddy-16) — first merge

• soup train --no-reexec printed a launch command with the user's own flags dropped, so following that line trained without --fsdp while still succeeding; rather than patch the printed copy they deleted it and derived the hint from the argv that actually launches the run (#415)

Shutaru (@Shutaru) — first two merges

• Kept the Transformers SFT import off the MLX dispatch route so backend: mlx cannot reach the PyTorch/TRL stack even if sft.py stops being import-light later — and, told the PR did not fix the defect its title claimed, retitled it to match reality rather than defending the framing, leaving #394 open (#431)
• Found the silent data corruption in assistant-only loss masking: BatchEncoding is not a dict, so the guard missed and the label mask was built from the mapping's key strings — no exception, normal loss curve, zero trained tokens (#439)

Harshit Sharma (@harshitthek) — first merge

• detect_device() did not know MLX, so an Apple Silicon run reported "CPU (no GPU detected)" and silently rewrote quantization: 4bit to none; the label was never the harm, so the quantization decision was extracted into resolve_quantization() and made explicit (#428)

Emmanuel Ziggah (@blackcoderx) — first two merges

• On Windows a process that genuinely exits with code 259 was indistinguishable from STILL_ACTIVE, so it read as alive forever — defeating reconcile-on-read and able to wedge the MCP execution cap shut with no error an operator could act on; disambiguated with WaitForSingleObject, and the two ~50-line copies of the check folded into one shared module (#436)
• soup data mix --optimize — the one command whose entire output is a config file — wrote one that would not parse, because data.train came out as a YAML list against a str field; then found the same defect a second time in --live's overlay and flagged it rather than widening the diff (#440)

Also reported, without a pull request: @lesterppo filed #433, the PEFT 0.19 adapter-materialization ambiguity behind two of the merges above, and #425.

Full changelog: https://github.com/MakazhanAlpamys/Soup/blob/main/CHANGELOG.md