feat(inference): add host-local vLLM serving presets

[prekshivyas]

Summary

Add the model-independent host-local vLLM serving contract required for optional single-host DGX Spark recipes. The change connects topology-free catalog selections to the existing managed local vLLM installer while preserving the automatic DGX Spark default.

No public model recipe, checkpoint, image, revision, or private release detail is included. A future public model requires a separately approved explicit-only recipe and preset.

Related Issue

Related to #8246.

Changes

• Register the vllm.host-local/v1 materializer and lifecycle descriptors.
• Allow managed inference presets that do not require a distributed topology.
• Resolve NEMOCLAW_SERVING_PRESET against a fresh single-host readiness report.
• Materialize a qualified selection into the existing managed local vLLM installer.
• Reject preset, model, and extra-argument conflicts before runtime side effects.
• Label the managed container with immutable preset and recipe identities and digests.
• Shell-quote catalog-derived vLLM arguments and environment values at the command boundary.

The serving catalog needs one topology-free adapter contract because the existing adapter requires distributed topology inputs. Direct installer flags would duplicate recipe settings in the installer and shell boundaries. The adapter, catalog, resolver, installer, vLLM command, and provenance-label tests protect this contract.

Type of Change

• Code change (feature, bug fix, or refactor)
• Code change with doc updates
• Doc only (prose changes, no code sample modifications)
• Doc only (includes code sample changes)

Quality Gates

• Tests added or updated for changed behavior
• Existing tests cover changed behavior — justification:
• Tests not applicable — justification:
• Docs updated for user-facing behavior changes
• Docs not applicable — justification: No shipped host-local recipe or preset, public CLI or configuration contract, default, documentation, Fern source, or managed-inference definition changed.
• Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
• Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: Independent maintainer review on 27be8a665 found no security findings across the nine repository security categories. Focused tests include shell-injection inputs, conflict rejection, gated-model access, and failure-before-effects behavior.
• Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Documentation Writer Review

• Documentation writer subagent reviewed the completed changes
• Result: no-docs-needed
• Evidence: Reviewed the 19-file TypeScript and test diff at 27be8a665 against cbabb66bf. No shipped host-local vLLM recipe or preset, public CLI or configuration contract, default, documentation, Fern source, or managed-inference definition changed. Existing automatic DGX Spark behavior and the documented two-DGX Spark preset remain unchanged. The change adds internal adapter, resolver, materialization, provenance-label, and shell-quoting support for a future separately approved recipe.
• Agent: Codex Desktop

DGX Station Hardware Evidence

• Tested on DGX Station
• Tested commit:
• Station profile/scenario:
• Result:
• Supporting evidence:

Verification

• PR description includes a Signed-off-by: line and every commit appears as Verified in GitHub
• Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run validate:pr passed after refreshing origin/main when hooks were skipped or unavailable
• Targeted behavior tests pass for the current change set, or tests are marked not applicable above — command/result or justification: npx vitest run --project cli for the seven affected test files passed 176 tests on 27be8a665; npm run typecheck:cli, npm run build:cli, and npm run checks:repository passed.
• Applicable broad gate passed — npm test for broad runtime/test-harness changes; npm run check for repo-wide validation/coverage changes — command/result: Not applicable. The focused source tests, CLI type-check, CLI build, repository checks, pre-commit hook, commit-msg hook, and pre-push hook cover this adapter and installer change.
• Quality Gates section completed with required justifications or waivers
• No secrets, API keys, or credentials committed
• npm run docs builds without warnings (doc changes only)
• Doc pages follow the style guide (doc changes only)
• New doc pages include SPDX header and frontmatter (new pages only)

Follow-up

• Persist and consume catalog provenance in host-local recovery and status receipts.
• Add a public single-host DGX Spark model recipe and preset only after its immutable artifacts and serving contract are approved and published.
• Run protected physical DGX Spark E2E for that future production recipe.

---

Signed-off-by: Prekshi Vyas prekshiv@nvidia.com

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

📝 Walkthrough

Walkthrough

The PR adds host-local DGX Spark vLLM catalog definitions, selection and materialization, topology-independent resolution, installer integration, provenance labels, and shell-safe vLLM command generation.

Changes

Host-local vLLM serving

Layer / File(s)	Summary
Catalog contracts and adapter registration src/lib/inference/serving/types.ts, src/lib/inference/serving/adapter-registry.ts, src/lib/inference/serving/catalog-loader.ts, src/lib/inference/serving/managed-cluster-*.ts, src/lib/inference/serving/*test.ts	The registry and catalog loader accept validated host-local vLLM definitions. Presets can omit topology bindings.
Selection materialization and topology-aware resolution src/lib/inference/serving/host-local-vllm-selection.ts, src/lib/inference/serving/resolver.ts, src/lib/inference/serving/types.ts, src/lib/inference/serving/*test.ts	Host-local selections validate recipe, platform, runtime, readiness, and model-length fields. Resolution omits topology qualifications for host-local recipes.
Installer integration and managed-cluster boundaries src/lib/inference/vllm.ts, src/lib/inference/serving/managed-cluster-installer.ts, src/lib/inference/serving/managed-cluster-materialize.ts, src/lib/inference/vllm.test.ts	The installer handles host-local selections separately from managed-cluster installation. vLLM profiles carry catalog provenance and emit Docker labels.
Shell-safe vLLM command generation src/lib/inference/vllm-models.ts, src/lib/inference/vllm-models.test.ts	The command builder validates serving environment names and shell-quotes model IDs, argument values, and environment values.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Catalog
  participant Resolver
  participant HostLocalSelection
  participant VllmInstaller
  participant Docker
  Catalog->>Resolver: provide host-local preset and recipe
  Resolver->>HostLocalSelection: materialize selected vLLM profile
  HostLocalSelection-->>Resolver: return model and catalog metadata
  Resolver->>VllmInstaller: install selected profile
  VllmInstaller->>Docker: run vLLM container with provenance labels

Loading

Possibly related issues

• [DGX Spark] Make single-host managed vLLM selectable through declarative serving presets #8246 — Covers declarative host-local DGX Spark vLLM selection through serving catalog and installer integration.

Possibly related PRs

• NVIDIA/NemoClaw#8129 — Both modify the managed-inference serving registry and catalog integration.
• NVIDIA/NemoClaw#8152 — This PR extends the serving catalog and resolver contracts introduced by that PR.
• NVIDIA/NemoClaw#8224 — Both extend adapter registry, catalog loader, and serving types for host-local recipes.

Suggested labels: area: install, area: architecture

Suggested reviewers: apurvvkumaria

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 11.90% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change: adding host-local vLLM serving presets.

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch agent/single-spark-vllm-presets

---

_{Comment @coderabbitai help to get the list of available commands.}

[github-code-quality]

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage in commit 27be8a6 in the agent/single-spark-v... branch remains at 96%, unchanged from commit c81f7fc in the main branch.

TypeScript / code-coverage/cli

The overall coverage in commit 27be8a6 in the agent/single-spark-v... branch is 81%. The coverage in commit cbabb66 in the main branch is 82%.

Show a code coverage summary of the most impacted files.

File	main cbabb66	agent/single-spark-v... 27be8a6	+/-
src/lib/inferen...ter-registry.ts	92%	84%	-8%
src/lib/inferen...talog-loader.ts	82%	80%	-2%
src/lib/inferen...er-installer.ts	88%	87%	-1%
src/lib/inference/vllm.ts	89%	89%	0%
src/lib/inferen.../vllm-models.ts	86%	86%	0%
src/lib/inferen...-materialize.ts	88%	88%	0%
src/lib/inferen...ing/resolver.ts	89%	89%	0%
src/lib/inferen...ter-executor.ts	70%	70%	0%
src/lib/core/version.ts	94%	98%	+4%
src/lib/inferen...lm-selection.ts	0%	81%	+81%

---

_{Updated August 05, 2026 17:00 UTC}

[github-actions]

PR Review Advisor — No blocking findings reported

Advisor assessment: No blocking advisor findings reported
Next action: No advisor follow-up needed.
Findings: 0 blockers · 0 warnings · 0 suggestions

Model lanes

• GPT-5.6 Terra (primary): Completed · medium confidence · 0 blockers · 0 warnings · 0 suggestions
• Nemotron 3 Ultra (second opinion): Failed

Second-opinion terminology and E2E selections are advisory. They do not change the primary assessment or E2E / PR Gate.

2 semantic terminology decisions

Terminology decisions are advisory. They affect the assessment only when a separate finding identifies concrete semantic impact.

• justified — host-local vLLM at src/lib/inference/serving/host-local-vllm-selection.ts:81: Keep “host-local vLLM” for the topology-free single-host contract.
• established — managed inference at src/lib/inference/serving/catalog-loader.ts:66: Keep the established controlled term with its existing meaning.

E2E guidance

Advisory only. E2E / PR Gate selects and runs jobs independently.

Recommended E2E: inference-routing, network-policy

Workflow run details

This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge.

[coderabbitai]

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/lib/inference/serving/adapter-registry.ts`:
- Around line 255-275: Introduce a distinct discriminated host-local recipe type
and use it in ResolvedHostLocalInferenceSelection instead of
ManagedInferenceServingRecipe, with distributed execution fields and
spec.bindings explicitly unavailable. Update
validateHostLocalVllmMaterializerRecipe and the related test to consume this
type directly and remove the current unknown/Record casts while preserving the
existing validation behavior.

In `@src/lib/inference/serving/managed-cluster-installer.ts`:
- Around line 434-437: Update the preview branch in the managed-cluster
installation flow around the topologyQualification check to return a
not-selected outcome for host-local presets instead of handled, allowing
runVllmInstall to continue into resolveHostLocalVllmSelection. Preserve the
revalidation branch’s fail-closed handled behavior, and add a regression test
covering a ready capability with an explicitly selected host-local preset.

In `@src/lib/inference/vllm.ts`:
- Around line 1706-1707: Update the hostLocalSelection branch in the inference
resolution flow to handle gated recipes before any Docker work occurs. When
recipe.spec.model.gated is true, invoke the existing assertGatedModelAccess
check before assigning resolved or proceeding to pullImage; otherwise preserve
the current host-local selection behavior.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 0a1e85a7-7dcb-4ac8-a282-84c65535c909

📥 Commits

Reviewing files that changed from the base of the PR and between 3f7097b and 91a06d3.

📒 Files selected for processing (15)

• src/lib/inference/serving/adapter-registry.test.ts
• src/lib/inference/serving/adapter-registry.ts
• src/lib/inference/serving/catalog-loader.test.ts
• src/lib/inference/serving/catalog-loader.ts
• src/lib/inference/serving/host-local-vllm-selection.ts
• src/lib/inference/serving/managed-cluster-installer.ts
• src/lib/inference/serving/managed-cluster-materialize.ts
• src/lib/inference/serving/resolver.test.ts
• src/lib/inference/serving/resolver.ts
• src/lib/inference/serving/types.ts
• src/lib/inference/serving/vllm-managed-support.ts
• src/lib/inference/vllm-models.test.ts
• src/lib/inference/vllm-models.ts
• src/lib/inference/vllm.test.ts
• src/lib/inference/vllm.ts

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/lib/inference/serving/types.ts`:
- Around line 118-138: Update HostLocalInferenceServingRecipe so omitted
host-local fields, including bindings and execution’s cluster-only properties,
use optional never types rather than required never properties. Adjust the
shared recipe input and related type guards to accept host-local recipes without
requiring structural compatibility with ManagedInferenceServingRecipe, while
preserving existing managed-cluster narrowing. Remove the resulting as unknown
as HostLocalInferenceServingRecipe and direct assertions, and ensure all
TypeScript CLI and plugin type-check configurations remain compatible.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 28f3fd1c-30b8-41a7-b7f9-bf641ea7df87

📥 Commits

Reviewing files that changed from the base of the PR and between 91a06d3 and 4c04de1.

📒 Files selected for processing (7)

• src/lib/inference/serving/adapter-registry.ts
• src/lib/inference/serving/managed-cluster-installer.test.ts
• src/lib/inference/serving/managed-cluster-installer.ts
• src/lib/inference/serving/resolver.ts
• src/lib/inference/serving/types.ts
• src/lib/inference/vllm.test.ts
• src/lib/inference/vllm.ts

🚧 Files skipped from review as they are similar to previous changes (3)

• src/lib/inference/serving/managed-cluster-installer.ts
• src/lib/inference/serving/resolver.ts
• src/lib/inference/vllm.ts

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[senthilr-nv]

Reviewed head 27be8a6 against base cbabb66. Product scope passes for the preparatory #8246 subset; all nine security categories pass with no findings. The 176 focused tests, CLI type-check, CLI build, repository checks, and hooks passed, and current automated review reports zero actionable findings.