feat(abi): embeddings/pooling through the ONE surface — LlamaModel arch, PoolingRunner in the engine step, vllm_embed (ABI v15), live /v1/embeddings

[localai-bot]

ONE-SURFACE fold ROW 6 (task #285, row ARCH-ONE-SURFACE leaf).

The engine-side pooler (ENG-POOLER-SEQ / ENG-POOLING-RUNNER) exists but is never invoked live. This row folds it onto the one surface:

• W1 registry+runner: register LlamaModel (upstream _EMBEDDING_MODELS, registry.py:230, as_embedding_model adapters.py:230) with is_pooling_model=true; the engine step routes pooling-model batches through the landed PoolingRunner instead of the sampler (model_runner.py:368-369, 1586-1607; scheduler.py:1718-1721). Refuse-by-task both directions.
• W2 ABI: vllm_embed + vllm_embedding_result(_free), VLLM_ABI_VERSION 14→15, floor pin advanced, strict-C references.
• W3 server: live /v1/embeddings (OpenAI shape), task-conditional registration, socket-level 404 pins both directions.
• W4 guard/records: abi-capability-allowlist embeddings row removed (leaves 1: mm-input); FEATURES row → reachable; spec ROW 6 progress; records.

Correctness anchor: the pooling lane's cosine gate extended with a registry-path arm — the engine-path vectors must be IDENTICAL to the direct PoolingRunner path and match the double-precision LAST+normalize reference, on a committed tiny synthetic LlamaModel fixture (the #121 precedent). Real-checkpoint re-verification is a named residual.

Spec: .agents/specs/embeddings-one-surface.md

🤖 Generated with Claude Code

https://claude.ai/code/session_014fZAcg1WcU8V629k6HWKys

[localai-bot]

Implementation complete at e4c29b3 (rebased onto main e484a63; keyed records re-applied on main wholesale).

Gates (CPU): fold gate test_llama_embedding_fold 4/4 cases / 231 asserts (direct registry path == f64 LAST+normalize reference; FULL-ENGINE path == direct path IDENTICAL vectors; chunked-prefill is_valid arm) · test_capi 48/48 / 462 (v15 floor pin, real fixture-checkpoint load through the public ABI, refuse-both-directions, argument contract) · test_dlopen 30/30 · test_openai_api_server 50/50 / 624 (embeddings shape + socket smoke + BOTH-direction 404 pins) · test_model_registry 24/24 / 820 · engine suites green (scheduler 423, llm_engine 204, engine_core 44, output_processor 77, qwen3_forward 1557, async_llm 342, llama_forward 509, loaded_engine_dense 65, scheduler_config 32, pooling_runner 14) · checker mutation suites green (routing 31, supported-models+surface-coverage 57) · scripts/agent-preflight.sh EXIT=0, all gates green.

Mutation kills (each RED then reverted, 9): ABI floor 15→14 (floor pin RED) · embed-on-text refusal deleted (capi RED ×3) · generation-on-pooling refusal deleted (refusal case hangs → SIGTERM FAILURE) · if (embedder_)→if (true) (text-server 404 pin RED) · generate route unconditional (embedding-server 404 pin RED) · engine-step pooling invocation deleted (fold gate RED ×2 cases) · scheduler pooling stop deleted (fold RED) · registry is_pooling_model=false (registry pin 819/820 + fold ×6 RED) · async-off wire dropped (fold engine arm RED + CHECK_FALSE(async) pin).

Allowlist diff: scripts/abi-capability-allowlist.txt embeddings/pooling row REMOVED — 1 row remains (multimodal input). example-abi-allowlist/ratchet UNTOUCHED (no example changes).

Residuals (honest): REAL embedding checkpoint (e5-mistral class) + vllm.LLM(task="embed").encode oracle cosine (committed synthetic fixture is the gate, the #121 precedent — no cosine-vs-oracle number fabricated); other _EMBEDDING_MODELS memberships (incl. MistralModel); /pooling+score/rerank/classify; matryoshka dimensions/base64/token-array inputs; vllm_embed batches sequentially through the synchronous engine (recorded deviation).

Note for the reviewer: the brief's premise that the pooling lane's gate ran "vs recorded oracle fixtures" was inaccurate — the existing gate is structural (double-precision reference; the qwen36_embed goldens are token-embedding LOOKUP goldens). Per the brief's own contingency, the gate was EXTENDED with the registry-path identity arm.

[localai-bot]

Landed on main as 57ed063 (mudler-authored squash, lease-guarded verified push). Fresh mutation review MERGE-CLEAN: every engine-step/scheduler pooling branch verified gated (text path byte-identical; ungated mutations red the text suites), both refusal guards load-bearing (removal reproduces real hangs), fold-gate arms independently killable, upstream cites verbatim, fixture byte-stable. Operator gates on the merged tree: six suites 6/6 SUCCESS, nine record gates green, FEATURES conflict composed with main's newer ROCm state. Embeddings are now embedder-reachable end-to-end: LlamaModel registry arch, live PoolingRunner, vllm_embed at ABI v15, /v1/embeddings. The capability tracker is at ONE remaining row (mm-input). The adjacent v13 vllm_complete_tokens guard fix rode along (confirmed pre-existing hang). Residuals as recorded: real-checkpoint cosine oracle, classify/score heads, matryoshka/base64 inputs, batched vllm_embed.