feat(security): add the NVD reconciliation library and the #7276 post-mortem#7370
Conversation
Public advisories can affect maintained packages weeks before they reach the reviewed corpus that npm audit enforces (#7276 timeline). Add an early-warning path and per-audit provenance so those windows become visible and provable from retained artifacts: - scripts/lib/advisory-early-warning.mts: dependency-free correlation of GitHub Security Advisory records (reviewed, unreviewed, and malware) against the reviewed-graph package inventories; exact npm name and parseable-range matches produce investigate signals, ambiguous or CPE-style matches stay informational and never block. - .github/workflows/advisory-early-warning.yaml: six-hourly paginated scan of all three advisory types, correlated against the inventory, reporting into one rolling security-labeled issue (marker- and author-verified, deduplicated by advisory id plus package). - scripts/audit-reviewed-npm-graph.mts and the wechat runtime audit action now write a *.provenance.json sidecar next to each raw report: scanner and npm/node versions, configured registry and derived bulk advisory endpoint, run start/end times, extracted advisory ids, and a failure marker whenever the audit attempt itself failed - including npm's parseable-error-JSON failure mode - so a failed attempt can never be mistaken for a comparable non-detection. - docs/security/advisory-early-warning.md: how the path works plus proposed defaults (scope, ownership, alert destination, SLA) pending maintainer confirmation. Refs #7338 Signed-off-by: Dongni-Yang <dongniy@nvidia.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
CodeRabbit: every source file carries the SPDX header; match the sibling docs/security file comment syntax. Refs #7338 Signed-off-by: Dongni-Yang <dongniy@nvidia.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Follow-up to the #7338 early-warning scan: use NVD as the supplementary reconciliation source the issue calls for, without ever treating NVD or CPE data as an authoritative npm mapping. - scripts/lib/nvd-reconciliation.mts: pure parser for NVD 2.0 records and per-signal reconciliation annotations (corroborated, nvd-missing, nvd-divergent) that are informational only - they never change a signal's action, confidence, or dedupe identity. - Advisory signals now carry the GHSA record's cve_id when present; the scan CLI accepts --nvd-records for offline attachment. - The scheduled workflow queries NVD for up to 20 signal CVE ids per run (7s spacing for the unauthenticated rate limit, cap logged when exceeded) and appends the annotation to the rolling issue lines. NVD being unreachable degrades to an "unavailable" note and never fails the run - deliberately softer than the fail-loud GitHub fetch, since NVD is supplementary. - docs: NVD reconciliation semantics plus the #7276 post-mortem section answering the issue's detection-trigger questions strictly from its own evidence table. Refs #7338 Signed-off-by: Dongni-Yang <dongniy@nvidia.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
📝 WalkthroughWalkthroughThe change adds validated CVE identifiers to advisory signals, introduces offline NVD 2.0 reconciliation, extends the scan CLI with inventory and NVD input files, adds comprehensive tests, and documents reconciliation behavior and post-mortem detection triggers. ChangesAdvisory NVD reconciliation
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant Operator
participant EarlyWarningScan
participant AdvisoryCorrelation
participant NvdReconciliation
participant NvdRecordsFile
Operator->>EarlyWarningScan: Run with --inventory and --nvd-records
EarlyWarningScan->>AdvisoryCorrelation: Correlate advisories against inventory
AdvisoryCorrelation-->>EarlyWarningScan: Signals with optional cveId
EarlyWarningScan->>NvdRecordsFile: Load pre-fetched responses
EarlyWarningScan->>NvdReconciliation: Attach NVD reconciliations
NvdReconciliation-->>EarlyWarningScan: Annotated signals
EarlyWarningScan-->>Operator: Print informational signal details
Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
Code Coverage OverviewLanguages: TypeScript TypeScript / code-coverage/pluginThe overall coverage in commit 389a282 in the TypeScript / code-coverage/cliThe overall coverage in commit 389a282 in the Show a code coverage summary of the most impacted files.
Updated |
PR Review Advisor — InformationalAdvisor assessment: Informational / high confidence Model lanes
Nemotron output stays in workflow artifacts and does not change the assessment above. E2E guidanceAdvisory only. E2E / PR Gate selects and runs jobs independently. Recommended E2E: None This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge. |
The source-shape scanner rightly flags assertions derived from repo config reads; give the scan CLI an explicit --inventory input so the CLI tests run against literal fixtures instead of the live reviewed inventory. The scheduled workflow keeps the repo-derived default. Refs #7338 Signed-off-by: Dongni-Yang <dongniy@nvidia.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Converting to draft: this PR is stacked on #7354, which was just narrowed per review — the scheduled early-warning workflow this PR extends moved to draft #7398, gated on product/security-owner sign-off on #7338. I'll restructure this PR the same way once the narrowing settles: the NVD reconciliation library and CLI plumbing can proceed against #7354's narrowed branch, and the workflow reconciliation step will fold into #7398. |
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
Resolve to the restructured non-operational slice: the NVD reconciliation library, scan-CLI plumbing (--nvd-records, --inventory, cveId), tests, and the reconciliation-semantics + #7276 post-mortem docs, rebased onto main after #7354 squash-merged. The scheduled workflow's NVD step and the policy-defaults doc section move to the sign-off-gated #7398, matching the #7354/#7398 split. Refs #7338 Signed-off-by: Dongni-Yang <dongniy@nvidia.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
🌿 Preview your docs: https://nvidia-preview-pr-7370.docs.buildwithfern.com/nemoclaw |
The operational half of the NVD reconciliation split (#7370 carries the library and CLI plumbing): the scheduled workflow queries NVD for up to 20 signal CVE ids per run (7 s spacing for the unauthenticated rate limit, cap logged when exceeded) and appends the agreement annotation to the rolling-issue lines. NVD being unreachable degrades to an "NVD: unavailable" note and never fails the run — deliberately softer than the fail-loud GitHub advisory fetch, since NVD is supplementary. Annotations extend line text only; the dedupe key is unchanged. Merge order: #7370 must land first — this workflow invokes the scan CLI's --nvd-records flag, which #7370 introduces. Refs #7338 Signed-off-by: Dongni-Yang <dongniy@nvidia.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Refs #7338 Signed-off-by: Dongni-Yang <dongniy@nvidia.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/security/advisory-early-warning.md`:
- Around line 43-49: Revise the added prose in the advisory documentation
sections, including the inventory description and the ranges noted in the
review, to use active voice addressed to the reader in present tense. Keep
exactly one complete sentence on each Markdown source line, preserving the
existing technical meaning and structured data notation.
In `@scripts/advisory-early-warning-scan.mts`:
- Around line 68-86: The loadInventoryOverride function silently skips malformed
inventory entries; add a warning whenever an entry has an invalid or missing
name or version before continuing. Include enough context to identify the
dropped entry and inventory file, while preserving acceptance of valid entries
and the existing fallback for origin.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 0c3c84b2-03b4-442e-bb59-55d42ec8e281
📒 Files selected for processing (6)
docs/security/advisory-early-warning.mdscripts/advisory-early-warning-scan.mtsscripts/lib/advisory-early-warning.mtsscripts/lib/nvd-reconciliation.mtstest/advisory-early-warning.test.tstest/nvd-reconciliation.test.ts
… doc style Review follow-up: a malformed --inventory entry now fails the scan run with an error naming the entry index instead of silently shrinking the inventory (which would under-report advisories with no visible signal), and the added doc sections follow the documentation style guide (one sentence per source line, direct imperatives for the CLI flags). Refs #7338 Signed-off-by: Dongni-Yang <dongniy@nvidia.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Refs #7338 Signed-off-by: Dongni-Yang <dongniy@nvidia.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>
apurvvkumaria
left a comment
There was a problem hiding this comment.
Security and merge-readiness review complete. The NVD path remains offline and informational, validates CVE identifiers and inventory input, does not alter enforcement or confidence, traverses nested configuration nodes without cycles, and has no unresolved review threads. All CI is green; targeted advisory/NVD coverage passes 93/93.
<!-- markdownlint-disable MD041 --> ## Summary Adds the canonical pre-tag `## v0.0.95` release entry to `docs/changelog/2026-07-24.mdx`, before the existing v0.0.94 entry. The entry summarizes approved user-visible changes merged since v0.0.94 and excludes internal-only prerequisites. ## Changes - Adds the v0.0.95 summary and detailed bullets for gateway lifecycle, recovery, state transfer, inference compatibility, sandbox security, Discord policy, and E2E evidence. - Links each user-facing theme to the most specific published documentation. - Records the release entry in the shared native changelog used by the OpenClaw, Hermes, and Deep Agents guides. Source summary: - [#7246](#7246), [#7228](#7228), [#7267](#7267), [#7489](#7489), [#7509](#7509), [#7351](#7351), and [#7290](#7290) -> `docs/changelog/2026-07-24.mdx`: Gateway authority, forward teardown and retry, managed recovery, Hermes restart recovery, scoped uninstall, and orphan-aware backup behavior. - [#7344](#7344) and [#7416](#7416) -> `docs/changelog/2026-07-24.mdx`: Atomic SQLite restore and host download verification. - [#7476](#7476), [#7347](#7347), [#7281](#7281), [#7485](#7485), [#7491](#7491), and [#7422](#7422) -> `docs/changelog/2026-07-24.mdx`: Windows Ollama reuse, CDI fallback, bounded OpenRouter connection setup, Nemotron-3 request compatibility, and managed Deep Agents retry and provider-error behavior. - [#6884](#6884), [#7481](#7481), [#6878](#6878), [#7467](#7467), [#7502](#7502), [#7503](#7503), [#7504](#7504), and [#7486](#7486) -> `docs/changelog/2026-07-24.mdx`: Trusted base-image overrides, local rebuild images, runtime validation, config preservation, reviewed package updates, and fewer final-image payload layers. - [#7303](#7303) -> `docs/changelog/2026-07-24.mdx`: Scoped Discord application-command management. - [#7488](#7488), [#7465](#7465), [#7497](#7497), [#7464](#7464), [#7501](#7501), [#7494](#7494), and [#7493](#7493) -> `docs/changelog/2026-07-24.mdx`: Selected-test risk signals, retry cleanup, full root-image validation, direct-main Hermes setup, executed PR-gate evidence, nightly history, and runner wait reporting. - [#7447](#7447) is an internal pinned-runtime prerequisite and is intentionally excluded from canonical supported-integration documentation. - [#7370](#7370) adds maintainer-only advisory reconciliation tooling and does not change supported user behavior. - [#7495](#7495) updates existing documentation and does not add a new v0.0.95 behavior claim. ## Type of Change - [ ] Code change (feature, bug fix, or refactor) - [ ] Code change with doc updates - [x] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates - [ ] Tests added or updated for changed behavior - [x] Existing tests cover changed behavior — justification: `test/changelog-docs.test.ts` validates the dated changelog structure, heading uniqueness, and published links. - [ ] Tests not applicable — justification: - [x] Docs updated for user-facing behavior changes - [ ] Docs not applicable — justification: - [ ] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [ ] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## Documentation Writer Review - [x] Documentation writer subagent reviewed the completed changes - Result: `docs-updated` - Evidence: `docs/changelog/2026-07-24.mdx`; writing rules, documentation style, factual release meaning, and published links reviewed at exact head `58b02f2bf`. - Agent: Codex documentation writer reviewer <!-- docs-review-head-sha: 58b02f2 --> <!-- docs-review-agents-blob-sha: 9c9b36d --> ## DGX Station Hardware Evidence - [ ] Tested on DGX Station - Tested commit: - Station profile/scenario: - Result: - Supporting evidence: ## Verification - [x] PR description includes a `Signed-off-by:` line and every commit appears as `Verified` in GitHub - [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or `npm run check:diff` passed when hooks were skipped or unavailable - [x] Targeted behavior tests pass for the current change set, or tests are marked not applicable above — command/result or justification: `npx vitest run test/changelog-docs.test.ts` passed 6 tests. - [ ] Applicable broad gate passed — `npm test` for broad runtime/test-harness changes; `npm run check` for repo-wide validation/coverage changes — command/result: - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) — the build passed with 0 errors and 2 Fern warnings. - [x] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) --- Signed-off-by: Senthil Ravichandran <senthilr@nvidia.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Added a new v0.0.95 changelog entry above v0.0.94. * Documented improved externally supervised gateway lifecycle ownership. * Improved snapshot restore reliability and SQLite state handling. * Tightened CLI `backup-all` behavior and host artifact verification. * Updated Windows onboarding guidance (including Ollama service reuse and CDI directory fallback). * Noted inference compatibility fixes, deeper agent failure classification, stricter base-image validation, updated Discord bot command permissions, and refined E2E release automation evidence handling. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Senthil Ravichandran <senthilr@nvidia.com>
Summary
The non-operational NVD slice of #7338: NVD as a supplementary reconciliation source (a pure library plus offline CLI plumbing) and the #7276 post-mortem answering the issue's "Questions to answer" — the post-mortem is an explicit #7338 acceptance checkbox.
Restructured 2026-07-24 to match the #7354/#7398 split: this PR was previously stacked on pre-narrowing #7354 and included the scheduled workflow's NVD step. That step (and the policy-defaults doc section) now belongs to the sign-off-gated #7398; this PR is rebased onto main and carries only pieces that define no operational surface — nothing here is scheduled, holds write permissions, or encodes policy. It can merge on normal review.
Changes
scripts/lib/nvd-reconciliation.mts(new, pure, dependency-free): parses NVD 2.0 API records (malformed → null, never throws; live-verified againstservices.nvd.nist.gov, including a 358-criteria CPE record and the reserved-id empty envelope) and reconciles each signal:corroborated/nvd-missing/nvd-divergent. Annotations are informational only — they carry no action or confidence and cannot alter a signal's dedupe identity (test-asserted), per the issue's requirement that ambiguous NVD/CPE matches never become authoritative npm mappings.AdvisorySignalgains an optionalcveIdfrom the GHSA record'scve_id; the scan CLI accepts--nvd-records <file>(offline attachment — the CLI never performs network requests) and--inventory <file>(hermetic test runs).fast-uri: reviewed-mapping delay, directly demonstrated;@opentelemetry/core: audit-coverage/execution-order gap; Jaeger: consistent but unproven;tar: unproven, evidence not retained. Post-mortem wording states plainly which covering mechanisms are merged and which ride the gated feat(security): schedule the advisory early-warning scan #7398.What moved to #7398
The scheduled workflow's NVD reconciliation step (rate-capped fetch,
NVD: unavailabledegradation, rolling-issue annotation) and the proposed policy defaults — operational surface under #7338's owner-sign-off gate.Documentation Writer Review
no-docs-neededscripts/lib/nvd-reconciliation.mtsandtest/nvd-reconciliation.test.tsnow traverse nested NVD configuration nodes as an internal correctness fix for the documented vulnerable-CPE parsing contract; no user-facing command, flag, configuration, output schema, or workflow changes.Testing
test/nvd-reconciliation.test.ts: 122 tests green across the three suite files locally; tsc strict clean on both.mtsentry points; biome, SPDX, test-size-budget, title-style, and source-shape guardrails verified locally.--list-packagesruns against the repo inventory.npx vitest run --project integration test/nvd-reconciliation.test.ts test/advisory-early-warning.test.tspasses 93/93 with nested NVD configuration coverage.Refs #7338 (the issue closes via #7398 once product/security owners record sign-off)
Signed-off-by: Dongni-Yang dongniy@nvidia.com
🤖 Generated with Claude Code
Summary by CodeRabbit
--inventory <file>for hermetic inventories and--nvd-records <file>to attach pre-fetched NVD reconciliation data without network access.#7276post-mortem.