Repository navigation
Project Health 2026 10 26
QASight already knows how the project it tests is doing: pass rates, flaky tests, tests that got slower, skipped and quarantined tests, the test pyramid. Each fact sits on its own page (dashboard, cycle page, flaky card), and there was no single answer to "how healthy are this project's tests?".
An AI review was also asked for, with review "skills" the user can see, switch on and off and expand. A first draft modeled the skills on the reviews run on QASight itself (architecture, UX copy, product readiness, accessibility). That was judged out of scope: those judge the product and its code, which is what code review tools such as SonarQube or CodeRabbit do. QASight runs tests and explains their results, so the AI part was cut down to checks about the tests only.
A Project health page (/health, in the main navigation) with two parts.
-
Test health score (
qasight/services/project_health.py,GET /api/v1/project-health?cycle=). A 0-100 score and a school grade, the weighted mean of six signals that have data: passing tests (30%), green builds over each cycle's last 20 runs (20%), stable tests (20%, each 1% flaky costs 10 points), speed trend (10%, each 1% slower costs 5), test hygiene (10%, each 1% skipped or quarantined costs 3) and the test pyramid (10%). It reusestest_history.list_cycle_testsand the pyramid model, so it agrees with the cycle page. Same runs, same score: no AI. The page also lists what to fix first (red cycles, flaky, slower, quarantined, skipped tests, each linked) and a score per cycle. Code coverage is not measured by QASight, so it is not a signal. -
AI review (
qasight/services/project_review.py). Seven built-in checks (GET /api/v1/project-health/skills): why tests are flaky, related failures, test strategy gaps, test debt, slow tests, test churn, and test names and messages (off by default). Each has a one-line tagline, a short summary, what it checks and what it reads, shown when its row is expanded, and a switch remembered in the browser.POST /api/v1/project-health/ai-reviewsends the enabled checks' rubrics and an evidence block (scores, per-cycle results, redacted failure messages, flaky and slow tests, test names, tests that appeared or disappeared) to the configured provider in one call, and expects JSON with a 0-100 score, a summary and findings per check. The overall AI score is the mean of the checks that scored. It never reads source code, and the page labels it an AI estimate.
A review is stored in the newest run it looked at (ai_project_review_v1 in the run's metadata, keyed by cycle or *), so it survives restarts on every database backend. GET /api/v1/project-health/ai-review returns it, stale when newer runs came in.
- One more page and four endpoints, all on the existing read model and AI provider. No new tables.
- The review needs a bigger answer than a run summary, so it raises the call's output tokens (450 per check, up to 4,000) and the timeout (at least 60 s), whatever AI settings say.
- A model that doesn't answer in JSON gives "unparseable_reply" and nothing is stored.
- The static demo exports the score, the checks and any stored review; running a review there is disabled like every other write.
- "Tests missing for recent changes" would need the changed files of each commit, which QASight doesn't record. It is left out for now.
Generated from docs/Specs & ADRs/Project Health - 2026-10-26.md. Edit it there; changes made in the wiki are overwritten.
Architecture
CLI Commands
Processes & Guides
- CI-CD Pipeline Setup
- Closed Network Install
- E2E Harness Operations Guide
- GitLab Pages Demo
- Publishing Docker Images
- Publishing to Artifactory
- Publishing to PyPI
- Setup Snippets
- Shared Resource Pools
- Stability and Deprecation Policy
- Team Server
Prompts & Snippets
Roadmap & Strategy
- Feature Gaps and Quick Wins - 2026-10-31
- New User Journey Audit - 2026-11-01
- Paid Features to Offer Free - 2026-10-31
- Product Roadmap
Specs & ADRs
- Adoption Blockers - 2026-11-12
- Affected Test Selection - 2026-10-30
- Alerts Where Developers Look - 2026-10-22
- Architecture Consolidation - 2026-10-06
- Architecture Review - 2026-10-31
- Automation and Unification - 2026-11-12
- BehaveX Reports - 2026-10-21
- Better Flaky Signal - 2026-11-01
- CLI-Only Bug Hunt - 2026-11-15
- Closed Network Install - 2026-11-15
- Command Adapter and JUnit Import - 2026-09-25
- Config JSON Schema - 2026-10-22
- Crash Detection - 2026-10-31
- Delta Comparison Policy
- Docs Site - 2026-10-19
- Docs Vault and Wiki Sync - 2026-10-07
- Easy Setup - 2026-11-01
- Engine Fixes - 2026-11-01
- Engineering Health Review - 2026-10-31
- Exporting Results - 2026-11-01
- External Audit Tools - 2026-11-11
- Flaky Test Issues - 2026-11-01
- Flaky Test Quarantine - 2026-10-22
- Free Copies of Paid Features - 2026-11-01
- GitHub Action PR Comment - 2026-10-19
- Human-Run Resource Provider - 2026-10-30
- Importer and Integration Bug Hunt - 2026-10-31
- Ingest and Team Server - 2026-10-22
- Interview Readiness Fixes - 2026-10-07
- Library Packaging
- Lifecycle Fixes - 2026-11-02
- MCP Server - 2026-11-01
- Menu by Question - 2026-11-12
- New User Journey Automation - 2026-11-01
- Open Source Features - 2026-10-19
- OpenTelemetry Export - 2026-10-30
- Phase 3 Delta Comparison Engine Plan
- Phase 3 Unified Dashboard Plan
- Phase 4 BYOK and Failure Analysis
- Phase 5 UI Redesign - Cycles-First Navigation
- Playwright Adapter - 2026-10-19
- Playwright JSON Import - 2026-10-22
- Plug and Play - 2026-11-12
- Plugin API - 2026-10-19
- Product Roadmap Work - 2026-10-25
- Product to Ten - 2026-10-25
- Project Health - 2026-10-26
- QASight Logo - 2026-11-05
- Qualflare Follow-ups - 2026-10-26
- Quarantine From the UI - 2026-11-01
- Specs & ADRs
- Release Gate - 2026-11-12
- Release Scorecard - 2026-11-12
- Rename to QASight - 2026-11-04
- Repository Pattern - Database-Agnostic Refactor
- Resource Lifecycle Hooks - 2026-10-26
- Resource Pools and Leases - 2026-10-26
- Run Deletion and Stop - 2026-11-02
- Runner Coverage - 2026-10-26
- Scale Runs and Stages - 2026-10-24
- Scorecard on MegaLinter - 2026-11-12
- Security and Install Review - 2026-10-31
- Standards Compliance - 2026-11-12
- Switchable Team Features - 2026-10-24
- Test Delivery Metrics - 2026-11-02
- Test Labels, Environments and Releases - 2026-11-02
- Test Source in AI Summaries - 2026-11-01
- Throwaway Stage Containers - 2026-10-24
- Tidy Cards and Demo Examples - 2026-11-05
- UI Design Review - 2026-10-31
- UI Polish - 2026-11-08
- UI Restyle - 2026-10-19
- UI Review Fixes - 2026-11-01
Testing Workflows
Vault