Skip to content

v3.15.7: Failed advisory lookups never read as clean; workers stop inventing data

Choose a tag to compare

@proffesor-for-testing proffesor-for-testing released this 15 Oct 10:52
· 15 commits to main since this release
7f2d091

What's New

AQE no longer reports a dependency scan as clean when the advisory lookup
failed, and its background workers no longer store invented CVEs, timings,
gate values or predictions. The quality gate, coverage, test retries, aqe init, learning backups and test generation also got fixes. Every change was
reviewed adversarially before merge.

Added

  • Dependency advisory scan (#1049, #1051): aqe security --dependencies and MCP security_scan_comprehensive with
    dependencies: true check your direct dependencies against OSV. The
    dependency list is sent to OSV only when you ask for this scan; default
    scans and background workers never contact it.
  • AQE_OSV_API_URL points the scan at another OSV endpoint. It must be
    https (http only for localhost, 127.x.x.x or [::1]). An invalid value
    fails only a requested dependency scan and names the setting. See
    docs/guides/dependency-advisory-scan.md.

Fixed

  • A failed advisory lookup is never a clean scan (#1049, #1051):
    • A network error, timeout, rate limit, server error or malformed reply now
      makes the scan failed or incomplete, with the error and the packages
      that could not be checked. It used to be recorded as "0 vulnerabilities",
      even when OSV was reachable.
    • The version that is checked is the one your lockfile resolved
      (package-lock.json, npm-shrinkwrap.json, pnpm-lock.yaml,
      yarn.lock), but only when the lockfile is in sync with package.json.
      A stale lockfile entry is reported as unchecked, never trusted.
    • Without a lockfile entry, only exact versions are checked. Ranges, tags
      and git, URL or tarball sources are reported as unchecked, never guessed.
    • OSV queries now send the version where OSV reads it. Before, OSV returned
      every advisory the package ever had, so counts were inflated (for
      example, PyPI django 3.2.0 showed 324 advisories instead of 66).
    • The output states the scope: only the direct dependencies of the scanned
      package.json, not transitive ones.
  • Background workers report only measured data (#1043, #1048): the
    security-scan, performance-baseline, quality-gate and defect-predictor
    workers no longer store invented findings (including made-up lodash and
    minimist CVEs), hardcoded timings, hardcoded gate values or invented
    defect predictions. With nothing measured they write nothing and say the
    data is unmeasured. Older invented records are ignored and replaced by the
    next measured run.
  • The quality gate rejects unmeasured evidence (#1040): an unmeasured
    count (-1, missing, NaN) no longer satisfies a max: 0 check, and a
    gate with no configured checks is an error instead of a 100-point pass.
  • Coverage (#1039, #1041):
    • Per-file coverage is found whatever path form you pass (src/a.ts or an
      absolute path), so defect prediction, defect investigation and quality
      assessment use the file's own measurement instead of the project-wide
      figure.
    • Coverage from sequential runs (parallel: false) is merged from the
      real hit data and matches vitest's own report. It used to average
      percentages and list the same source file twice.
  • Test retries (#1035): a Vitest retry passes only when the exact file
    you asked for passed. A sibling file whose name starts the same way can no
    longer clear a failure.
  • Blocked scratch folder (#1038): when neither the private cache folder
    nor its fallback can be used, AQE explains why and suggests
    AQE_PROJECT_ROOT, instead of failing with a bare EEXIST/ENOTDIR.
  • Learning backups (#1034): two aqe learning backup runs in the same
    second no longer write the same file, so neither backup is silently lost.
  • Test generation (#1029, #1032): test_generate_enhanced with
    saveResults: true now writes the generated test files, not just the
    manifest and report.
  • aqe init (#1037): honours AQE_PROJECT_ROOT and writes all its
    files into that project, including a repository that sits in a shared
    folder such as /tmp (sticky-bit 1777). It refuses a root that doesn't
    exist.
  • Dependencies: yaml is updated to 2.9.1, which fixes a parser hang on
    crafted input.
  • README: monthly and total download badges restored.

Changed

  • aqe security --dependencies exits 3 when the scan is incomplete (some
    advisories could not be checked), 1 for high or critical findings,
    2 for medium findings and 0 when it is complete and clean.
  • A workspace (monorepo) root is reported incomplete: its own
    dependencies are scanned, but member dependencies are not scanned yet
    (#1057). Scan each member directory instead; a member walks up to the
    root lockfile and can be complete.
  • For library users of agentic-qe/shared: the OSVClient list methods
    throw OSVLookupError instead of returning [] on failure (use
    lookupBatch / lookupNpmDependencies for partial results), and
    DependencyScanResult gains status, measured, packagesChecked and
    uncheckedPackages.
  • Quality gate results gain optional evidence fields.
  • Default aqe learning backup file names now include milliseconds and a
    unique id.

Known issues

  • Workspace member dependencies are not scanned from a monorepo root yet
    (#1057).
  • Coverage for package targets in a workspace doesn't use the configured
    workspace root yet (#1036, fix in review).
  • Five of the seven quality-gate metrics still have no command that records
    them (#1009).
  • Duration comparisons are noisy under machine load (#1044).
  • The MCP defect_predict tool doesn't record a prediction, so the
    defect-predictor worker stays unmeasured after it (#1050).
  • Some other lookups still turn a failure into "no findings": an
    unreachable DAST target, deprecation and upgrade checks (#1052).
  • Path follow-ups: retry evidence ignores symlinked paths, and per-file
    coverage lookups resolve against the current folder (#1047).
  • Windows paths are unit-tested only, not run on Windows (#971, #982).
  • Partly fixed and still open: #972, #1020, #1030, #1031; other
    follow-ups: #1022, #1024.

Getting Started

npx agentic-qe init --auto

See CHANGELOG for full details.