Skip to content

v3.13.1: Honest measurement — coverage, LLM wiring, and a consumer CVE

Choose a tag to compare

@proffesor-for-testing proffesor-for-testing released this 23 Jul 16:43
1751132

What's New

A correctness and honesty release. Four reported issues, three of which share one
root cause: a tool told you something it had not actually established.

Coverage analysis no longer reports numbers it never measured (#569)

On a Rust crate, coverage_analyze_sublinear returned 78.3% lines, 100% branches
and 0% functions — none of it measured. It flagged #[cfg(test)] blocks as
untested production code, cited line numbers past the end of the file, and
attached a confidence score to all of it.

Rust crates are now measured with cargo llvm-cov when available. Test code is
excluded from production coverage. A metric that was not collected reports null
rather than 0 or 100. When nothing could be measured, the result says so
(estimated: true, measured: false) instead of presenting a guess as data.

test_generate_enhanced actually calls your LLM (#567)

The tool advertised AI-powered generation, but the LLM router never reached the
code path it runs — so it returned the same generic scaffold regardless of your
provider configuration. The only symptoms were a suspiciously constant coverage
estimate and a sub-20ms response.

It now routes through your configured model, and reports llmEnhanced so a silent
fall back to templates is visible. A broken provider (bad key, timeout) reports
the fallback honestly instead of claiming enhancement it did not perform.

A HIGH-severity advisory no longer reaches consumers (#565)

@huggingface/transformers was an optional peer dependency — but npm installs
optional peers, so every install pulled in onnxruntime-node → adm-zip and its
advisory. It is now an explicit opt-in for anyone who wants local in-process
embeddings. A clean-room install of this release audits 0 vulnerabilities,
down from 4 HIGH.

Per-agent LLM routing you can configure on disk (#568)

Route individual agent types through different providers — a strong reasoning
model for security agents, a free local model for mechanical ones — without
changing your global default:

// .agentic-qe/llm-config.json
{
  "defaultProvider": "ollama",
  "agentOverrides": {
    "qe-security-scanner": { "provider": "claude-code", "model": "opus" },
    "qe-mutation-tester":  { "provider": "ollama" }
  }
}

QE skills for the Codex CLI

aqe init --with-codex now installs three Codex-native skills
(aqe-test-change, aqe-review-quality, aqe-plan-quality), Codex lifecycle
hooks, and an AGENTS.md. If you drive your project with Codex rather than
Claude Code, the QE workflows are available there natively.

Breaking change

coverage_analyze_sublinear now returns null — not 0 — for any coverage
metric that was not collected. lineCoverage, statementCoverage,
branchCoverage and functionCoverage are number | null.

  • 0 means a measured zero
  • null means unknown

Callers doing arithmetic or comparisons on these fields should handle null;
check the new measured / estimated flags to tell the cases apart. This is
deliberately a visible break rather than a silent wrong number — coercing "we
don't know" to 0 is what produced the impossible 100%/0% pair in #569.

Getting Started

npx agentic-qe init --auto

See the release notes and CHANGELOG
for full details, and ADR-126
for the principle behind the coverage changes.