feat(cli): larql run <vindex3-container> — the container speaks for itself - #382
Merged
Conversation
… itself run() detects a VINDEX3 container from its index generation and routes to run_cmd_vindex3: tokenizer from the container, greedy decode through the same interpreter `vindex3 exec` reports on, text streamed, EOS from generation_config.json. The setup run_exec owned (inspect, plan, operand store, backend choice) moves to vindex3_cmd::prepare so both verbs share one path; the greedy loop moves to vindex3_cmd::decode with a token sink, and run_generate returns the ids. Dense-engine flags are refused by name. --emit-ids makes a run an id-level oracle: identical to exec on oute-mamba2attn-250m (CPU) and qwen3-4b (Metal).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
larql run <vindex3-container> [prompt]— text in, text out, from the container's own program.run()detects a VINDEX3 container from itsindex.jsongeneration before any VINDEX2-only reader opens it and routes torun_cmd_vindex3: the container's tokenizer, greedy decode through the same interpreterlarql vindex3 execreports on, text streamed as it is produced, EOS from the container'sgeneration_config.json. Weights load once; every prompt (and every chat line) gets a fresh continuation state.--metalselects the Metal realisation (macOS,gpufeature); otherwise thelarql-computeCPU kernels.--top,--kv-cache,--engine,--ffn,--routed-from,--moe-*,--experts*,--ops,--constrained,--image,--mm-weights) are refused by name, in one message, never ignored.--emit-idsprints prompt and generated ids to stderr, so a run doubles as an id-level oracle forvindex3 exec --tokens.tokenizer.jsonis refused with a pointer tovindex3 exec --tokens.Structure
The setup
run_execowned (inspect → plan → operand store → backend choice) moves tovindex3_cmd/prepare.rsbehind aBackendVisitor(backends are distinct concrete types, so the choice is made once and the visitor sees the type). The greedy loop moves tovindex3_cmd/decode.rswith a token sink that can halt the turn;run_generateis now the exec verb's report wrapper and returns the ids. Note:prepareis a second opener besidelarql_inference::vindex3::runtime::open_component(#381 made that one policy-aware); the stacked PR deletes this duplicate and delegates — that is the step that makes the opener the single authority.Witness (release build, "The capital of France is", 12 tokens)
vindex3 exec --tokens--metalGates
21 new tests (loop, dispatch, detection, refusals, EOS by id and by stop string, chat-turn isolation, Metal on the fixture).
cargo fmt --all --check; clippy--bins --tests --no-deps -- -D warningsunder default features and--no-default-features;cargo test -p larql-cliin both shapes (805 CPU-only);registry check; doc link/reference gates. Coverage in the CI shape:decode.rs100%, run arm 91.5%+,prepare.rs82% (the untested arm is the pre-existing "plan does not close" refusal, which needs binary-segment surgery to reach).Docs
docs/cli.md(run section + exec table),docs/vindex3-registry-design.md(the recorded gap is closed), README (one sentence). The stale "no KV cache" statement onexec --generateis corrected.