Repository navigation
v3.15.5: honest coverage and quality results
·
565 commits
to main
since this release
What's New
Honest coverage and quality results: AQE no longer reports a fake 0%, approves
without evidence, or saves a coverage estimate as if it were measured. Also
fixes aqe code index measuring the wrong folder.
Fixed
aqe code indexmeasures the real project (#994, #997). The project
root is now found from the target you pass: the nearest folder with
package.json,.git,go.mod,Cargo.tomlorpyproject.toml, walking up.
aqe code index .at a monorepo root measures the root. Previously it often
measuredsrc/or.agentic-qe/workersand reported 0 tests. It never falls
back to the filesystem root.- Coverage analysis on a subfolder or an existing report (#995, #1000):
- a subfolder target (
src/) runs the tests from the project root and reports
only the files under the target, instead of a "measured" 0%; - an existing report (
lcov.info,coverage-final.json,
coverage-summary.json, Cobertura XML) can be passed as the target; - when nothing measured the target, the result is unmeasured, not 0%. A
file that ran with 0 hits is still a real 0%; - nothing is written into your project (
coverage/,.vitest/, results
files); reports go to a temp folder that is always removed; - the test runner is started without a shell or
npx(nothing is
downloaded), and its whole process group is stopped on timeout or when the
MCP server exits, so no test processes are left running; aqe coverage -F jsonprints only JSON; metrics a report never collected
(for example branch data) show as n/a and don't count against the threshold.
- a subfolder target (
quality_assessno longer approves without evidence (#996, #1001).
When a requested metric is unmeasured, the result is
status: insufficient-evidencewith the missing metrics listed, and it
escalates instead of approving. A measured metric that fails its threshold
blocks even when others are unmeasured. Request and configured thresholds are
honoured. The fixed security score of 85 is gone (unmeasured is reported as
unmeasured).- Coverage estimates are never saved as measured evidence (#1008, #1013).
Static estimates, runs where the runner was missing, crashed or timed out, and
coverage data submitted by callers are no longer stored ascoverage:latest
or quality evidence. A background job that overwrote the last measurement
with an empty record every few seconds is fixed. - Security:
sharpupdated to 0.35.5 (#993).
Changed
aqe coverageexits 3 when the target is unmeasured, and 0 (not 1)
for a report without branch or function data that meets the threshold on the
metrics it has.aqe qualityexits 3 for insufficient evidence;aqe ci's quality
artifact uses the sameinsufficient-evidencestatus.quality_assessvalidatesmetrics(a non-empty list of known names) and
threshold(0–100). Ungated static approvals report confidence 0.7 at most.- Coverage persistence is fail-closed: programmatic callers of the coverage
analysis API must passmeasured: truefor results to be stored.
Known issues
- Five of the seven quality-gate metrics have no command that records them yet,
soquality_assess --runGatecan't pass for most projects (#1009). - The coverage runner's Windows path is unit-tested only, not run on Windows.
Getting Started
npx agentic-qe init --autoSee CHANGELOG for full details.