Add production-grade QA analysis and reporting - #1
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b0264ebe9c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| matches := 0 | ||
| for _, file := range files { | ||
| candidate := normalizedChangePath(file.Path) | ||
| if strings.HasSuffix(candidate, "/"+wanted) || strings.HasSuffix(wanted, "/"+candidate) { |
There was a problem hiding this comment.
Avoid matching shorter coverage paths to changed files
When a changed file is packages/api/src/service.go and the report only has coverage for src/service.go, this reverse-suffix condition treats that unrelated shorter path as the changed file's coverage. In that case --fail-diff-coverage can pass using hits from another file instead of reporting missing evidence for the changed file; the import-prefix case only needs the coverage path to end with the repo-relative path, or else stronger disambiguation is needed.
Useful? React with 👍 / 👎.
| if failQuality == 0 { | ||
| failQuality = cfg.FailQuality |
There was a problem hiding this comment.
Honor an explicit zero quality threshold
When .test-cli.json sets failQuality (as this repo now does), passing --fail-quality 0 is parsed as the same zero default and this block restores cfg.FailQuality. Since gateExitCode treats 0 as disabled, users cannot temporarily disable the configured quality gate from the CLI; track whether the flag was provided instead of using zero as both “unset” and “disabled.”
Useful? React with 👍 / 👎.
What changed
Why
The tool previously normalized tests and coverage but did not provide a standard diagnostic model, repeated-run evidence, changed-code risk, or production-grade protection against stale artifacts and untrusted report roots. These changes make the output actionable for QA, CI, and AI-agent consumers across supported ecosystems.
Impact
Users get schema
test-cli/report@2, richer static and runtime diagnostics, regression/flaky/change gates, and a self-contained report site with multiple coverage and quality views. Real executions now reject missing per-language test results and cannot silently reuse unchanged artifacts.Validation
make lint testgo test -race -count=1 ./...go mod verifyactionlint .github/workflows/ci.ymlshellcheck -S warning scripts/*.shandbash -n scripts/*.sh