[REVIEW] dependency-scanning: add lockfile integrity and registry signature evidence

[heickerv1001-dev]

Skill Being Reviewed

Skill name: dependency-scanning
Skill path: skills/appsec/dependency-scanning/SKILL.md

False Positive Analysis

Benign dependency tree that should not be reported as supply-chain tampering by lockfile presence alone:

{
  "packages": {
    "node_modules/example": {
      "version": "1.2.3",
      "resolved": "https://registry.npmjs.org/example/-/example-1.2.3.tgz",
      "integrity": "sha512-<redacted-sri>",
      "license": "MIT"
    }
  }
}

Why this is a false positive risk:

The current skill correctly tells reviewers to commit lockfiles and analyze the full resolved dependency tree. But a reviewer can still over-score a healthy lockfile as risky if they see tarball URLs or integrity hashes without understanding that npm lockfiles intentionally record the resolved artifact location and Subresource Integrity value for reproducible installs.

The review should distinguish normal lockfile metadata from genuinely risky cases: missing integrity, unexpected external tarball hosts, git dependencies without immutable commits, registry drift, unsigned packages where signatures are expected, or lockfile/manifests that disagree.

Coverage Gaps

Missed variant 1: package-lock entry lacks integrity or resolves from an unexpected host

{
  "packages": {
    "node_modules/lodash": {
      "version": "4.17.21",
      "resolved": "https://example-cdn.invalid/lodash-4.17.21.tgz"
    }
  }
}

Why it should be caught:

The skill says to read lockfiles, but its output template does not require reviewers to record artifact source and integrity evidence. npm's package-lock format includes resolved and integrity fields for package artifacts. If a registry package resolves to an unexpected third-party host, lacks an integrity hash, or changes artifact location unexpectedly, that is a supply-chain risk even when the version number is not vulnerable by CVE.

Missed variant 2: git dependencies are pinned only to branch/tag-like refs

{
  "dependencies": {
    "internal-tool": "git+https://github.com/example/internal-tool.git#main"
  }
}

Why it should be caught:

A dependency scan that only records package name and version can miss mutable source dependencies. For git dependencies, reviewers should verify that the lockfile resolved commit is immutable and that the manifest does not rely on a moving branch or tag for production builds. Otherwise the dependency tree can change without a version bump, CVE, or lockfile visibility that the current template would surface.

Missed variant 3: npm registry signatures and provenance are not checked where available

npm audit signatures
npm audit signatures --json --include-attestations

Why it should be caught:

The skill recommends tools such as npm audit and SBOM generators, but it does not mention npm audit signatures. npm documents registry signature verification and provenance-attestation verification through that command. For high-risk supply-chain reviews, signature/provenance status is different evidence from CVE status: a package can have no known CVEs while still lacking expected artifact signature evidence.

Missed variant 4: install-script presence is listed but not tied to lockfile evidence

{
  "packages": {
    "node_modules/postinstall-runner": {
      "version": "0.1.0",
      "resolved": "https://registry.npmjs.org/postinstall-runner/-/postinstall-runner-0.1.0.tgz",
      "integrity": "sha512-<redacted-sri>",
      "hasInstallScript": true
    }
  }
}

Why it should be caught:

The skill mentions reviewing npm install hooks under typosquatting, but package-lock already exposes hasInstallScript. Reviewers should record which transitive packages have install scripts, whether they are production dependencies, whether scripts run in CI/release builds, and whether the organization uses controls such as script approval/deny lists or isolated builds. This is not necessarily a vulnerability by itself, but it is important supply-chain evidence.

Edge Cases

• A resolved tarball URL is normal for npm packages and should not be a finding unless the host, scheme, registry, or package identity is unexpected for the project.
• registry.npmjs.org can appear as npm lockfile shorthand/magic registry value; reviewers should not misreport it as a missing URL.
• Some private registries legitimately proxy public packages; record the expected registry policy before flagging external resolution.
• Signature/provenance verification may be unavailable for older packages or non-npm ecosystems. Report it as not available or not evaluated, not as a CVE-equivalent failure.
• Install scripts may be legitimate for native modules, but production release builds should still know which scripts can execute and from which dependencies.

Remediation Quality

• Fix resolves the vulnerability
• Fix doesn't introduce new security issues
• Fix doesn't break functionality
• Issues found: Add lockfile artifact-integrity review gates for npm and equivalent ecosystems. Require reviewers to record package source (resolved/registry), integrity/checksum, git commit immutability, install-script presence, signature/provenance verification status, and any mismatches between manifest, lockfile, SBOM, and registry metadata.

Comparison to Other Tools

Tool / Source	Catches this?	Notes
npm package-lock metadata	Yes	Exposes exact dependency tree metadata, artifact source, integrity, and install-script flags.
npm audit signatures	Yes	Verifies registry signatures and provenance attestations for downloaded packages where supported.
CycloneDX / SPDX SBOMs	Partial	Can record component identity and hashes, but the reviewer must verify that SBOM identity matches lockfile and artifact evidence.
CVE scanners	No	CVE scanners can report known vulnerabilities while missing mutable refs, missing integrity, unexpected tarball hosts, or unsigned artifacts.
Current skill	Partial	Requires lockfiles and SBOMs, but does not turn lockfile source/integrity/signature evidence into review output fields.

Overall Assessment

Strengths:

• Strong coverage of CVSS/EPSS/KEV triage, license risk, transitive dependency risk, typosquatting, and SBOM generation.
• Good prompt-injection safety: it tells reviewers not to execute dependency files or package metadata.
• Good multi-ecosystem manifest discovery across Node, Python, Go, Java, Rust, Ruby, and PHP.

Needs improvement:

• The skill should treat lockfiles as artifact provenance evidence, not just as a way to enumerate versions.
• The output template should include integrity/source/signature fields so non-CVE supply-chain risks are visible.
• The npm-specific guidance should use package-lock fields such as resolved, integrity, and hasInstallScript, plus signature/provenance verification where available.
• Mutable git dependencies and external tarball dependencies need separate severity guidance from normal registry packages.

Priority recommendations:

1. Add a Lockfile Artifact Integrity subsection after Transitive Dependency Risk.
2. Add grep/read checks for resolved, integrity, hasInstallScript, git+, http://, file:, link:, npm-shrinkwrap.json, and private-registry hostnames.
3. Extend the Supply Chain Risk Indicators section with: missing integrity, unexpected artifact host, mutable git ref, external tarball, install-script production dependency, signature/provenance not verified, and manifest/lockfile/SBOM mismatch.
4. Add npm-specific command evidence for npm audit signatures and npm audit signatures --json --include-attestations, while preserving the skill constraint not to execute package scripts.

Sources Checked

• npm Docs, package-lock.json: https://docs.npmjs.com/cli/v11/configuring-npm/package-lock-json/
• npm Docs, npm audit and Audit Signatures: https://docs.npmjs.com/cli/v11/commands/npm-audit/
• CycloneDX Specification overview: https://cyclonedx.org/specification/overview/
• SPDX Specification: https://spdx.github.io/spdx-spec/

Bounty Info

• I have read and agree to the CONTRIBUTING.md bounty terms.
• Preferred payment method: Crypto; details can be provided privately after acceptance.

[Alex-Parejo]

Implemented in PR #387: #387

[heickerv1001-dev]

Follow-up: PR #387 is still open and ready for review. I rechecked the issue state today; if anything else is needed for acceptance, I can address it.