Skip to content

test(onboard): add an in-memory machine runtime fixture - #8546

Merged
jyaunches merged 1 commit into
NVIDIA:mainfrom
JulienAu:test/onboard-machine-runtime-fixture
Aug 7, 2026
Merged

test(onboard): add an in-memory machine runtime fixture#8546
jyaunches merged 1 commit into
NVIDIA:mainfrom
JulienAu:test/onboard-machine-runtime-fixture

Conversation

@JulienAu

@JulienAu JulienAu commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Summary

Five onboarding-machine suites each declared the same in-memory OnboardRuntime over a session store, and eight duplicated the deep cloneSession helper byte for byte. This is sequential PR 2 of issue #8289: it moves both into test/helpers/onboard-machine-runtime-fixture.ts and migrates the eight suites. The deterministic clock is overridable where assertions depend on a specific timestamp, and flow-slices keeps its intentionally inert markStepFailed and completeSession behavior local rather than growing the fixture behavior options.

Related Issue

Refs #8289 (sequential PR 2 of 4; PR 1 is #8537)

Changes

  • Add test/helpers/onboard-machine-runtime-fixture.ts: cloneSession, createTestRuntime(initialSession?, { now? }), and a re-export of the session helpers the suites consume. The test/helpers/ placement follows the existing shared-fixture convention (base-image-test-harness, messaging-conflict-fixtures, env-test-helpers) and keeps the source-architecture fan-in budget for onboard-session.ts untouched; the eight suites drop their direct runtime imports of onboard-session in the process. Requirement and consumers: the eight machine suites below. A direct change cannot remove the duplication because each suite owns an inline copy; the protecting tests are the migrated suites themselves. The fixture holds no filesystem or global state, so the PR 1 review convention about registering workspace cleanup does not apply here.
  • Migrate runner, runner-sequence, sequence-runner, flow-sequence, and flow-phases/agent-policy-finalization to the shared runtime; the two clock families keep their asserted timestamps at their call sites.
  • Migrate runtime, flow-slices, and transition-traces to the shared cloneSession only.
  • Every migrated file has a negative line delta (net -139 across the PR).

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Quality Gates

  • Tests added or updated for changed behavior
  • Existing tests cover changed behavior — justification: test-only consolidation; the eight migrated suites are the protected behavior and pass unchanged (63 tests).
  • Tests not applicable — justification:
  • Docs updated for user-facing behavior changes
  • Docs not applicable — justification: test fixture only; no user-facing surface or documented behavior changes.
  • Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
  • Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification:
  • Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Documentation Writer Review

  • Documentation writer subagent reviewed the completed changes
  • Result: no-docs-needed
  • Evidence: test-only fixture and suite migrations; no user-facing surface changes. Docstrings on the new fixture reviewed against the writing rules.
  • Agent: Claude Code

DGX Station Hardware Evidence

  • Tested on DGX Station
  • Tested commit:
  • Station profile/scenario:
  • Result:
  • Supporting evidence:

Verification

  • PR description includes a Signed-off-by: line and every commit appears as Verified in GitHub
  • Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run validate:pr passed after refreshing origin/main when hooks were skipped or unavailable — ran npm run validate:pr; every hook passes except hadolint, which reports a pre-existing info-level finding (Dockerfile:1543 SC2015) that is byte-identical on unmodified main; this PR changes no Dockerfile
  • Targeted behavior tests pass for the current change set, or tests are marked not applicable above — command/result: npx vitest run --project cli on the eight migrated suites; 8 files, 63 tests pass in normal and shuffled order. tsc -p tsconfig.cli.json and npm run checks:repository pass.
  • Applicable broad gate passed — npm test for broad runtime/test-harness changes; npm run check for repo-wide validation/coverage changes — command/result:
  • Quality Gates section completed with required justifications or waivers
  • No secrets, API keys, or credentials committed
  • npm run docs builds without warnings (doc changes only)
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only)

Signed-off-by: JulienAu 16043912+JulienAu@users.noreply.github.com

Summary by CodeRabbit

  • Tests
    • Standardized onboarding flow and runtime tests on a shared test fixture.
    • Improved test isolation through consistent session cloning, persistence, lifecycle handling, and event simulation.
    • Added configurable fixed timestamps for more deterministic test execution.

Five machine suites each declared the same in-memory OnboardRuntime
over a session store, and eight duplicated the deep cloneSession
helper byte for byte. Move both into
test/helpers/onboard-machine-runtime-fixture.ts, which also fronts the
session helpers so the suites drop their direct onboard-session imports; the
deterministic clock is overridable where assertions depend on a
specific timestamp, and flow-slices keeps its intentionally inert
markStepFailed and completeSession behavior local. Every migrated
file has a negative line delta.

Refs NVIDIA#8289

Signed-off-by: JulienAu <16043912+JulienAu@users.noreply.github.com>
@copy-pr-bot

copy-pr-bot Bot commented Aug 7, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@coderabbitai

coderabbitai Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

The pull request adds a shared in-memory onboarding runtime fixture. Eight onboarding machine test files now use its session helpers and runtime factory instead of local implementations.

Changes

Onboarding runtime fixture

Layer / File(s) Summary
Shared runtime implementation
test/helpers/onboard-machine-runtime-fixture.ts
Adds session cloning, helper re-exports, deterministic timestamps, in-memory persistence, safe updates, lifecycle handling, and OnboardRuntime construction.
Runtime-oriented test migration
src/lib/onboard/machine/flow-phases/agent-policy-finalization.test.ts, src/lib/onboard/machine/flow-sequence.test.ts, src/lib/onboard/machine/runner-sequence.test.ts, src/lib/onboard/machine/sequence-runner.test.ts
Replaces local runtime dependency setup and session cloning with createTestRuntime.
Session helper migration
src/lib/onboard/machine/flow-slices.test.ts, src/lib/onboard/machine/runner.test.ts, src/lib/onboard/machine/runtime.test.ts, src/lib/onboard/machine/transition-traces.test.ts
Imports session types, constants, creation, cloning, and safe-update helpers from the shared fixture.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Suggested labels: refactor

Suggested reviewers: ericksoa, apurvvkumaria, cv

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the added shared in-memory onboarding machine runtime fixture.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/helpers/onboard-machine-runtime-fixture.ts`:
- Around line 29-31: Update cloneSession so the parsed deep copy remains the
fallback when normalizeSession returns null; replace the original session
fallback with the serialized copy, preserving cloneSession’s state isolation.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 2439a835-9ac4-4813-9c7e-7f2c0e0b4fea

📥 Commits

Reviewing files that changed from the base of the PR and between d8fbb6e and 18924fa.

📒 Files selected for processing (9)
  • src/lib/onboard/machine/flow-phases/agent-policy-finalization.test.ts
  • src/lib/onboard/machine/flow-sequence.test.ts
  • src/lib/onboard/machine/flow-slices.test.ts
  • src/lib/onboard/machine/runner-sequence.test.ts
  • src/lib/onboard/machine/runner.test.ts
  • src/lib/onboard/machine/runtime.test.ts
  • src/lib/onboard/machine/sequence-runner.test.ts
  • src/lib/onboard/machine/transition-traces.test.ts
  • test/helpers/onboard-machine-runtime-fixture.ts

Comment thread test/helpers/onboard-machine-runtime-fixture.ts
@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor — Informational

Advisor assessment: Informational / low confidence
Next action: No advisor follow-up needed.
Findings: 0 blockers · 0 warnings · 0 suggestions
Status: PR review advisor failed: PR review advisor SDK execution failed: session: 429 status code (no body); turn: terminology-review-analysis: 429 status code (no body)

Model lanes

  • GPT-5.6 Terra (primary): Failed
  • Nemotron 3 Ultra (second opinion): Skipped

Second-opinion terminology and E2E selections are advisory. Live E2E does not run automatically for pull requests.

E2E guidance

Advisory only. A maintainer can dispatch the default E2E suite against this exact revision.

Recommended E2E: cloud-onboard

Workflow run details

This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge.

@jyaunches
jyaunches merged commit 237459c into NVIDIA:main Aug 7, 2026
12 of 14 checks passed
jyaunches added a commit that referenced this pull request Aug 7, 2026
<!-- markdownlint-disable MD041 -->
## Summary

`cloneSession` from #8546 returned the caller-owned session when
normalization rejected its serialized copy. This follow-up keeps the
serialized copy as the fallback so fixture mutations cannot alter
caller-owned top-level or nested state.

## Changes

- Use the serialized session copy as both the normalization input and
fallback.
- Add a regression test with an invalid session version. The test proves
that normalization rejection still returns independent top-level and
nested state.
- Record the detection gap: #8546 exercised valid sessions only, so
normalization always succeeded and did not reach the aliasing fallback.
- Close the complete #8546 review set. The CodeRabbit isolation finding
is fixed. The docstring-coverage warning is advisory because local
exports have JSDoc and re-export documentation remains source-owned. The
PR Review Advisor reported no findings and failed only because its model
request returned HTTP 429.
- Inspect the equivalent local clone helpers in
`test/helpers/onboard-final-flow-phases.ts`,
`src/lib/onboard/resume-machine-repair.test.ts`,
`src/lib/onboard/runtime-boundary-step-result.test.ts`, and
`src/lib/onboard/runtime-boundary.test.ts`. Their callers construct
valid sessions and do not expose #8546's independent-mutation contract,
so this PR does not change them.

## Type of Change

- [x] Code change (feature, bug fix, or refactor)
- [ ] Code change with doc updates
- [ ] Doc only (prose changes, no code sample modifications)
- [ ] Doc only (includes code sample changes)

## Quality Gates

- [x] Tests added or updated for changed behavior
- [ ] Existing tests cover changed behavior — justification:
- [ ] Tests not applicable — justification:
- [ ] Docs updated for user-facing behavior changes
- [x] Docs not applicable — justification: this changes an internal test
fixture and its regression coverage. No CLI, configuration, output,
workflow, default, API, policy schema, or supported user behavior
changes.
- [ ] Sensitive paths changed (security, policy, credentials, preflight,
onboarding, inference, runner, sandbox, or messaging)
- [ ] Sensitive-path review completed or maintainer-approved waiver
recorded — reviewer/approval link/justification:
- [ ] Non-success, skipped, or missing CI check accepted by maintainer —
check name, approval link, and follow-up issue:

## Documentation Writer Review

- [x] Documentation writer subagent reviewed the completed changes
- Result: `no-docs-needed`
- Evidence: the completed diff changes only
`test/helpers/onboard-machine-runtime-fixture.ts` and
`test/helpers/onboard-machine-runtime-fixture.test.ts`. The reviewer
found no user-visible behavior or documentation impact and approved the
test title and existing JSDoc ownership.
- Agent: Codex Desktop
<!-- docs-review-head-sha: 64c19cf -->
<!-- docs-review-agents-blob-sha: c69aad4 -->

## DGX Station Hardware Evidence

- [ ] Tested on DGX Station
- Tested commit: Not applicable; `scripts/prepare-dgx-station-host.sh`
does not change.
- Station profile/scenario: Not applicable.
- Result: Not applicable.
- Supporting evidence: Not applicable.

## Verification

- [x] PR description includes a `Signed-off-by:` line and every commit
appears as `Verified` in GitHub
- [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or
`npm run validate:pr` passed after refreshing `origin/main` when hooks
were skipped or unavailable
- [x] Targeted behavior tests pass for the current change set, or tests
are marked not applicable above — command/result or justification: `npx
vitest run --project integration
test/helpers/onboard-machine-runtime-fixture.test.ts` passed 1/1; the
eight #8546 CLI suites passed 63/63; `npm run typecheck:cli` passed;
`npm run test:projects:check` reported exact membership for 2,231
candidate files across seven projects.
- [ ] Applicable broad gate passed — `npm test` for broad
runtime/test-harness changes; `npm run check` for repo-wide
validation/coverage changes — command/result: not applicable. The change
updates one fixture fallback and adds one focused test; targeted
validation covers every #8546 consumer.
- [x] Quality Gates section completed with required justifications or
waivers
- [x] No secrets, API keys, or credentials committed
- [ ] `npm run docs` builds without warnings (doc changes only)
- [ ] Doc pages follow the [style
guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md)
(doc changes only)
- [ ] New doc pages include SPDX header and frontmatter (new pages only)

---
Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Improved session cloning so copied sessions remain independent even
when normalization fails.
* Prevented changes to a cloned session from affecting the original
session.

* **Tests**
* Added regression coverage for cloning invalid-version sessions and
nested steps.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>
jyaunches pushed a commit that referenced this pull request Aug 7, 2026
<!-- markdownlint-disable MD041 -->
## Summary

Four installer suites each declared the same sourced-installer spawn
with small env, PATH, and timeout variations, and two duplicated the npm
stub with divergent ci handling. This is sequential PR 3 of issue #8289:
it moves the sourced-body runner, the npm stub builder, and a disposable
checkout scaffold into `test/helpers/installer-run-fixture.ts` and
migrates six pilot suites, which keep their scenario prefixes, stub
snippets, and environment values at the call sites through one-line
wrappers.

## Related Issue

Refs #8289 (sequential PR 3 of 4; PR 1 is #8537, PR 2 is #8546)

## Changes

- Add `test/helpers/installer-run-fixture.ts`:
`runInstallerSourcedBody(body, { home?, homePrefix?, extraEnv?,
includeNodeOnPath?, timeoutMs? })` consolidating the four spawn copies
behavior for behavior (SIGKILL applies exactly when a timeout does, and
the environment stays minimal with no process.env spread, as in every
copy), `writeNpmStub(fakeBin, { installSnippet?, handleCi? })`
preserving each family's unexpected-invocation guard (`handleCi`
reproduces the preflight family's `npm ci` handling without weakening
the other family's stricter stub), and `createInstallerCheckout` for the
bin-plus-prefix scaffold. It builds on the existing
`installer-sourced-env.ts` constants rather than duplicating them.
- Migrate `install-preflight`, `install-build-dependency-preflight`,
`install-express-wsl-ollama`, `install-station-controller-binding`,
`install-station-pair-preparation`, and
`install-station-vllm-continuation`. Every migrated file has a negative
line delta.
- Ratchet the `ci/test-file-size-budget.json` entry for
`install-preflight.test.ts` from 3,921 to its new 3,908-line size, as
the size gate requires after shrinking a legacy file.
- Register test-finished cleanup at every migrated wrapper that creates
a temporary HOME, while preserving caller ownership when an existing
HOME is supplied.

## Type of Change

- [x] Code change (feature, bug fix, or refactor)
- [ ] Code change with doc updates
- [ ] Doc only (prose changes, no code sample modifications)
- [ ] Doc only (includes code sample changes)

## Quality Gates

- [x] Tests added or updated for changed behavior
- [ ] Existing tests cover changed behavior — justification:
- [ ] Tests not applicable — justification:
- [ ] Docs updated for user-facing behavior changes
- [x] Docs not applicable — justification: test fixtures only; no
user-facing surface or documented behavior changes.
- [ ] Sensitive paths changed (security, policy, credentials, preflight,
onboarding, inference, runner, sandbox, or messaging)
- [ ] Sensitive-path review completed or maintainer-approved waiver
recorded — reviewer/approval link/justification:
- [ ] Non-success, skipped, or missing CI check accepted by maintainer —
check name, approval link, and follow-up issue:

## Documentation Writer Review

- [x] Documentation writer subagent reviewed the completed changes
- Result: `no-docs-needed`
- Evidence: Test-only installer fixture refactor, caller-owned HOME
preservation, and helper-owned HOME cleanup coverage; no user-visible
surface changed.
- Agent: Codex Desktop
<!-- docs-review-head-sha: ac123e9 -->
<!-- docs-review-agents-blob-sha: 12ad395 -->

## DGX Station Hardware Evidence

- [ ] Tested on DGX Station
- Tested commit:
- Station profile/scenario:
- Result:
- Supporting evidence:

## Verification

- [x] PR description includes a `Signed-off-by:` line and every commit
appears as `Verified` in GitHub
- [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or
`npm run validate:pr` passed after refreshing `origin/main` when hooks
were skipped or unavailable — `npm run check:diff` passes on the
current-main refresh, including repository checks, secret scan, commit
lint, and pre-push type checking
- [x] Targeted behavior tests pass for the current change set, or tests
are marked not applicable above — command/result: the completed helper
and six-suite aggregate passes 203/203 tests; current-main revalidation
passes the four affected suites 105/105 plus helper ownership coverage
2/2. Isolated cleanup verification leaves no helper-owned HOME
directories. CLI typecheck, repository checks, formatting, structural
gates, commit hooks, and pre-push hooks pass.
- [ ] Applicable broad gate passed — `npm test` for broad
runtime/test-harness changes; `npm run check` for repo-wide
validation/coverage changes — command/result:
- [x] Quality Gates section completed with required justifications or
waivers
- [x] No secrets, API keys, or credentials committed
- [ ] `npm run docs` builds without warnings (doc changes only)
- [ ] Doc pages follow the [style
guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md)
(doc changes only)
- [ ] New doc pages include SPDX header and frontmatter (new pages only)

---
Signed-off-by: JulienAu <16043912+JulienAu@users.noreply.github.com>


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Tests**
* Improved installer test reliability with shared utilities for isolated
environments, temporary checkouts, configurable package-manager
behavior, output handling, and timeouts.
* Standardized installer test setup while preserving existing scenarios,
coverage, and simulated failures.
* Added coverage for temporary environment cleanup and preservation of
caller-provided settings.
* Updated the legacy test file size budget to reflect current coverage.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: JulienAu <16043912+JulienAu@users.noreply.github.com>
Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>
Co-authored-by: JulienAu <16043912+JulienAu@users.noreply.github.com>
Co-authored-by: Apurv Kumaria <akumaria@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants