Skip to content

fix(e2e): restore CLI artifact for security posture - #8621

Merged
cv merged 1 commit into
mainfrom
codex/restore-security-posture-cli-artifact
Aug 8, 2026
Merged

fix(e2e): restore CLI artifact for security posture#8621
cv merged 1 commit into
mainfrom
codex/restore-security-posture-cli-artifact

Conversation

@apurvvkumaria

@apurvvkumaria apurvvkumaria commented Aug 8, 2026

Copy link
Copy Markdown
Collaborator

Summary

The trusted security-posture job now restores the exact candidate CLI artifact produced by generate-matrix before telemetry, installation, and live testing. This supplies nemoclaw/dist/shared/sandbox-name.cjs while retaining the existing receipt, provenance, digest, and security-posture gates.

Changes

  • Restore the pinned, content-addressed CLI artifact immediately after the no-build prepare-e2e step in security-posture.
  • Register security-posture as an exact-artifact consumer and require sandbox-name.cjs at both producer and restore contract boundaries.
  • Update workflow tests, runner-comparison ordering coverage, and the internal E2E telemetry description.
  • Root cause: prepare-e2e skipped the CLI build for security-posture, while the artifact-consumer registry also excluded that job even though generate-matrix packaged the required shared module. The former contract test encoded that exclusion, so it did not catch the missing restore. The failure is visible in current-main run 31243363889.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Quality Gates

  • Tests added or updated for changed behavior
  • Existing tests cover changed behavior — justification:
  • Tests not applicable — justification:
  • Docs updated for user-facing behavior changes
  • Docs not applicable — justification: No user-facing behavior changes; the internal test/e2e/README.md was updated to match the trusted workflow ordering.
  • Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
  • Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: Independent frozen-diff security and correctness review passed for 4c2e1db90; receipt, provenance, credential scope, failure ordering, and security-posture controls remain intact.
  • Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Documentation Writer Review

  • Documentation writer subagent reviewed the completed changes
  • Result: docs-updated
  • Evidence: Updated test/e2e/README.md to document artifact restoration before telemetry and the resulting security-posture measurement window.
  • Agent: Codex Desktop

DGX Station Hardware Evidence

  • Tested on DGX Station
  • Tested commit:
  • Station profile/scenario:
  • Result:
  • Supporting evidence:

Verification

  • PR description includes a Signed-off-by: line and every commit appears as Verified in GitHub
  • Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run validate:pr passed after refreshing origin/main when hooks were skipped or unavailable
  • Targeted behavior tests pass for the current change set, or tests are marked not applicable above — command/result or justification: Four focused E2E-support files passed 78 tests; the origin/main path-scoped selection passed 397 tests across 36 files.
  • Applicable broad gate passed — npm test for broad runtime/test-harness changes; npm run check for repo-wide validation/coverage changes — command/result:
  • Quality Gates section completed with required justifications or waivers
  • No secrets, API keys, or credentials committed
  • npm run docs builds without warnings (doc changes only)
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only)

Signed-off-by: Apurv Kumaria akumaria@nvidia.com

Summary by CodeRabbit

  • Bug Fixes

    • Improved security-posture end-to-end workflow reliability by restoring the exact CLI artifact before telemetry and tests run.
    • Ensured required shared CLI components are included during packaging and restoration.
    • Updated workflow validation to enforce consistent artifact and telemetry initialization behavior.
  • Documentation

    • Clarified artifact restoration and telemetry initialization timing in end-to-end testing guidance.

Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>
@apurvvkumaria apurvvkumaria self-assigned this Aug 8, 2026
@coderabbitai

coderabbitai Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 5aafd7e1-d4e0-4d3f-9067-7954ab4219e1

📥 Commits

Reviewing files that changed from the base of the PR and between 5722e63 and 4c2e1db.

📒 Files selected for processing (7)
  • .github/workflows/e2e.yaml
  • test/e2e/README.md
  • test/e2e/support/cli-artifact-workflow-boundary.test.ts
  • test/e2e/support/runner-comparison-workflow-boundary.test.ts
  • tools/e2e/cli-artifact-workflow-boundary.mts
  • tools/e2e/prepare-e2e-workflow-boundary.mts
  • tools/e2e/runner-comparison-workflow-boundary.mts
💤 Files with no reviewable changes (2)
  • tools/e2e/prepare-e2e-workflow-boundary.mts
  • tools/e2e/runner-comparison-workflow-boundary.mts

📝 Walkthrough

Walkthrough

The E2E workflow now restores the exact-commit CLI artifact for security-posture. Boundary validation requires sandbox-name.cjs and checks artifact consumers. Runner-comparison documentation and tests now place telemetry initialization after artifact restoration.

Changes

CLI artifact alignment

Layer / File(s) Summary
Shared artifact contract and validation
tools/e2e/cli-artifact-workflow-boundary.mts, test/e2e/support/cli-artifact-workflow-boundary.test.ts
The contract includes security-posture as a CLI artifact consumer. Producer and restore validation require sandbox-name.cjs. Tests cover missing shared modules and exact artifact restoration.
Security-posture restoration and initialization
.github/workflows/e2e.yaml, tools/e2e/prepare-e2e-workflow-boundary.mts, tools/e2e/runner-comparison-workflow-boundary.mts, test/e2e/support/runner-comparison-workflow-boundary.test.ts, test/e2e/README.md
The security-posture job restores the exact-commit artifact before telemetry initialization and tests. Boundary expectations and telemetry documentation reflect this order.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

Suggested labels: release-target, v0.0.105

Suggested reviewers: cv, ericksoa, jyaunches

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes restoring the CLI artifact for the security-posture E2E job.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Create stacked PR
  • Commit on current branch
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/restore-security-posture-cli-artifact

Comment @coderabbitai help to get the list of available commands.

@github-code-quality

github-code-quality Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage in commit 4c2e1db in the codex/restore-securi... branch remains at 96%, unchanged from commit 5722e63 in the main branch.

TypeScript / code-coverage/cli

The overall coverage in commit 4c2e1db in the codex/restore-securi... branch remains at 81%, unchanged from commit c6fb802 in the main branch.

Show a code coverage summary of the most impacted files.
File main c6fb802 codex/restore-securi... 4c2e1db +/-
src/lib/core/pr...mpt-activity.ts 92% 67% -25%
src/lib/policy/...ateway-state.ts 73% 56% -17%
src/lib/policy/index.ts 62% 63% +1%
src/lib/inferen...er-lifecycle.ts 76% 77% +1%
src/lib/adapters/http/probe.ts 86% 87% +1%
src/lib/credentials/store.ts 55% 57% +2%
src/lib/inferen...tion-cluster.ts 83% 85% +2%
src/lib/policy/...scope-render.ts 94% 97% +3%
src/lib/trace.ts 90% 94% +4%
src/lib/inferen...vllm-api-key.ts 83% 90% +7%

Updated August 08, 2026 14:56 UTC

@github-actions

github-actions Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor — No blocking findings reported

Advisor assessment: No blocking advisor findings reported
Next action: No advisor follow-up needed.
Findings: 0 blockers · 0 warnings · 0 suggestions

Model lanes

  • GPT-5.6 Terra (primary): Completed · high confidence · 0 blockers · 0 warnings · 0 suggestions
  • Nemotron 3 Ultra (second opinion): Failed

Second-opinion terminology and E2E selections are advisory. Live E2E does not run automatically for pull requests.

2 semantic terminology decisions

Terminology decisions are advisory. They affect the assessment only when a separate finding identifies concrete semantic impact.

  • established — exact-commit CLI artifact at .github/workflows/e2e.yaml:5171: Keep the established term.
  • justified — shared CLI modules at test/e2e/support/cli-artifact-workflow-boundary.test.ts:931: Keep the modifier because the test title identifies the restored module requirement.

E2E guidance

Advisory only. A maintainer can dispatch the default E2E suite against this exact revision.

Recommended E2E: cloud-inference, cloud-onboard, security-posture

Workflow run details

This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge.

@cv
cv merged commit 37f5475 into main Aug 8, 2026
64 of 67 checks passed
@cv
cv deleted the codex/restore-security-posture-cli-artifact branch August 8, 2026 14:58
@github-actions github-actions Bot added the v0.0.106 Release target label Aug 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

v0.0.106 Release target

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants