Skip to content

fix(xtask): narrow scancode partial-success reuse#594

Merged
mstykow merged 5 commits intomainfrom
fix/npm-cli-verify-followups
Apr 9, 2026
Merged

fix(xtask): narrow scancode partial-success reuse#594
mstykow merged 5 commits intomainfrom
fix/npm-cli-verify-followups

Conversation

@mstykow
Copy link
Copy Markdown
Owner

@mstykow mstykow commented Apr 9, 2026

Summary

  • narrow compare-outputs so ScanCode is only reused after a non-zero exit when it still produced valid JSON that clearly encodes file-level non-fatal scan errors
  • suppress bogus root identity fields for unnamed npm v1 lockfiles and cover that parser behavior with both focused unit and golden tests
  • rewrite the npm/cli benchmark entry as an end-state comparison, including the verified 4-process compare context and the concrete Provenant advantages over ScanCode

Scope and exclusions

  • Included:
    • xtask/src/bin/compare_outputs.rs partial-success validation tightening
    • npm lock unnamed-root parser behavior and parser-only regression coverage
    • docs/BENCHMARKS.md and scorecard wording cleanup for the verified npm/cli result
  • Explicit exclusions:
    • no broader npm/yarn/pnpm parser changes beyond unnamed-root v1 lock identity
    • no new compare run beyond the already recorded npm/cli verification artifacts
    • no change to the general benchmark table format outside the npm/cli row

Intentional differences from Python

  • This PR does not try to match ScanCode's broad non-zero-exit behavior. It intentionally keeps Provenant's compare wrapper fail-closed except for the narrowly validated case where ScanCode still produced usable JSON with explicit file-level scan_errors.

Follow-up work

  • Created or intentionally deferred:
    • follow-up npm-family verification targets such as yarnpkg/berry, oven-sh/bun, vercel/next.js, and microsoft/vscode remain future work under the now-verified row

Expected-output fixture changes

  • Files changed: testdata/npm/package-lock-v1-unnamed-root.json, testdata/npm/package-lock-v1-unnamed-root.json.expected.json
  • Why the new expected output is correct:
    • unnamed v1 lockfiles can legitimately contain dependency graphs without a root package identity, so Provenant should preserve the dependency extraction while omitting bogus root name/version/purl fields

Verification

  • cargo test npm_lock
  • cargo test --release --features golden-tests test_golden_npm_lock_v1_unnamed_root
  • cargo test --manifest-path xtask/Cargo.toml --bin compare-outputs validate_scancode_output_on_failure
  • npm run check:docs

Compare-run notes

  • Verified npm/cli compare artifact: .provenant/compare-runs/20260409T124932Z-cli-3457
  • Cached ScanCode source reused for that verification flow: .provenant/compare-runs/20260409T113235Z-cli-99118/raw/scancode.json
  • Provenant advantages documented in the benchmark row: successful root workspace manifest scanning where ScanCode hit workspace-assembly scan errors, no bogus empty-root lockfile identities, and successful completion of all but one large registry-fixture JSON that timed out under ScanCode

mstykow and others added 5 commits April 9, 2026 15:59
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
@mstykow mstykow merged commit 53593ff into main Apr 9, 2026
14 checks passed
@mstykow mstykow deleted the fix/npm-cli-verify-followups branch April 9, 2026 14:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant