Rust port: why the rewrite is incoherent, and the two triggers that would make the hybrid worth revisiting (from #775)

[Ercaner1988]

Moved here from #775 at @MakazhanAlpamys' request — "Please move it to a Discussion; converting it to issues now would create work with no trigger." Nothing is being asked for. The point of writing it down is that the analysis is cheaper to reopen than to redo, and the triggers that would justify reopening it are worth naming while they are still absent.

Position up front: a Rust rewrite of Soup is incoherent, and even the hybrid track has no trigger today. The measurement in the last section — which I ran to test the one gate that was cited in #775 — argues against the Rust track and for a plain Python fix instead.

Why the rewrite is not on the table

Soup's value is that it wraps the Python ML stack: torch, transformers, peft, trl, bitsandbytes, accelerate, mlx. Porting the trainer surface (SFT plus the preference trainers, QLoRA, FSDP) means either reimplementing that stack on candle / burn, which do not currently cover it, or paying FFI costs stacked on top of the trl/transformers API churn this project already absorbs. Both are worse than what exists.

So the only version worth discussing is a hybrid: leave training in Python, move some non-training layer to Rust.

The hybrid layers, honestly ranked

Layer	Today	Rust equivalent	Verdict
Config validation	pydantic v2, config/schema.py (7,053 lines)	serde + validators	Best fit on paper, worst fit in practice — see below
Data tooling	JSONL / datasets	polars + Arrow	Real speedup available, no complaint driving it
CLI shell	typer	clap	Attractive only if startup is the problem — measured below, it is not a Rust problem
Hub HTTP	huggingface-hub	hf-hub	Viable, no benefit worth a second HTTP stack
Lazy ML stack	function-local imports	cargo features	The current design already achieves this

config/schema.py inverts on inspection. It is the largest single module and the most mechanical, which is what makes serde look attractive — but it is also the surface that tracks trl/transformers kwargs, i.e. the highest-churn file in the repo. Porting it would move the churn across an FFI boundary and duplicate the source of truth CONTRIBUTING names as single ("These models are the single source of truth for valid fields and defaults"). It is the worst candidate, not the best.

The two triggers

@MakazhanAlpamys named both in #775, and they are the right gates:

1. Single-binary distribution becomes a product goal. It is not on the roadmap. Nothing in the repo builds a binary; there is no Cargo.toml and no maturin in pyproject.toml, so a hybrid would also introduce a build toolchain and a wheel matrix to a currently pure-Python distribution.
2. CLI startup becomes a measured problem that Python cannot fix. Tested below.

If neither fires, this stays closed. If (1) fires, the order is CLI shell first, then data tooling; config stays in pydantic.

Testing the startup gate — the answer is a Python fix, not a Rust one

#775 cites tests/test_cli_startup_is_light.py as governing startup on the Python side. Read closely, that guard pins something narrower than "startup is fine": it asserts that heavy ML modules do not leak into import time (torch, transformers, mlx, …), across soup_cli.cli and 50+ command modules. It does not assert a wall-clock budget, so the project's own modules can get expensive without the guard noticing. So I measured.

python -c pass            min   86 ms   median   95 ms
import soup_cli.cli       min 1215 ms   median 1673 ms
soup --help               min 1480 ms   median 1511 ms
soup version              min 1287 ms   median 1389 ms

-X importtime, largest cumulative subtrees:

soup_cli.cli
  soup_cli.commands.autopilot
    soup_cli.autopilot.analyzer -> soup_cli.autopilot.generate_config
      soup_cli.config.schema        <- the dominant share
  rich.console
  typer

Caveats, because these numbers decide the argument. One machine, Windows, Python 3.12, python -m soup_cli rather than the console script, in a uv-provisioned environment; 5 runs, min and median reported. Windows process startup is slower than Linux, so the absolute figures are an upper bound and are not a CI-comparable measurement. -X importtime inflates totals (its own tree sums to ~4.1 s against a ~1.2–1.7 s wall clock), so only the relative shares above should be read, not those durations. Someone on Linux posting the same two commands would sharpen this considerably.

What survives the caveats: soup version costs roughly 1.3 s against an ~90 ms interpreter floor, and the largest identified share is Soup importing its own 7,053-line pydantic schema through commands.autopilot — not typer, not rich, and no ML dependency. The heavy-import guard is doing its job; this cost is below its threshold by design.

That points at deferring soup_cli.autopilot behind a function-local import in the same style CONTRIBUTING already prescribes for the ML stack, not at clap. A rewrite that shaved the typer/rich share while config.schema still loaded eagerly would spend a language migration on the smaller half of the problem.

I have not filed that as an issue — it is a measurement from one Windows box, and it belongs to whoever wants to confirm it on Linux CI first. Happy to open one with a startup-budget test alongside the fix if that is wanted; equally happy for it to stay here.

Refs #775.

[Ercaner1988]

Measured on real runners now, and the number I quoted here needs correcting: soup version is 0.41–0.67 s on ubuntu/macos/windows-latest against a 13–25 ms interpreter floor — not the ~1.3 s I measured on my own Windows box, whose interpreter floor alone is ~86 ms. Read my earlier figure as an upper bound on slow hardware.

The structure of the finding held on all three platforms: import soup_cli.config.schema alone is 240–300 ms, roughly half of import soup_cli.cli and 4–8× typer plus rich.

One thing I got wrong here, worth flagging because it was the actionable half: I suggested this was a function-local import away. It is not. -X importtime blames commands.autopilot only because it imports config.schema first — a module-level import walk shows 7 of cli.py's 87 direct dependencies reach it (autopilot, data, data_canary, data_doctor, data_topics, sweep, train), so deferring one just makes the next importer pay.

Filed with the full tables, the import graph and a repro as #780. It does not change this thread's conclusion — the startup gate for a Rust port stays unmet, and the remaining cost is Soup's own pydantic model tree, which clap would not touch.

[Ercaner1988]

Closing this as resolved — measured rather than argued, and nothing here asks anything of Soup.

Soup already gets Rust where Rust pays. Resolving core + train for Linux x86_64 / Python 3.12 gives 81 packages. By each wheel's WHEEL generator: 4 are Rust (maturin): tokenizers, safetensors, hf-xet, pydantic-core, i.e. tokenization, weight I/O, Hub transfer and config validation. 35 are C/C++/CUDA (torch, triton, the nvidia-* stack, numpy, pyarrow…) and 42 are pure-Python CLI plumbing. The one measured startup cost (#780, config/schema.py) sits on top of pydantic-core, which is already Rust: the cost is Python-side model construction, so "move the dependency to Rust" has already happened there without removing it.

Also checked: swapping stdlib sqlite3 for Turso (pyturso 0.7.2). It is Alpha, ships no Windows wheel, and on Windows cannot open a database that another process holds open (Locking error (os error 33)), whether that process uses Turso or stdlib sqlite3. Soup's tracker is written by soup train while ui/tui/mcp read it from other processes, so that is a hard stop today. Multi-process WAL sits behind an experimental flag; Linux untested.

Your two gates stand unchanged. Reopen if either fires.