Test Plan Comparison

Test Plan Comparison (Modernized) — Issue #1957

Screen: gui/templates/results/planComparison.html BFF: api/plancomparison/index.php ASIDE: Reports → Test Plan Comparison (new entry, $TLS_link_report_plan_comparison, present in all 19 server locales) Action: $actions->planComparison in lib/functions/common.php

Purpose

Every TestLink 1.9.20 report is scoped to one test plan. Comparing two plans meant opening two report tabs and comparing the numbers by eye — pass rates, failed cases, blocked cases, defect references, coverage — and the comparison was never written down.

This screen is a net-new 2.0.1 report: it puts two test plans of the same test project side by side, shows the delta (B - A) per metric and which plan is better, and breaks the results down per build and per platform. The screen is read only — it never writes an execution.

What it shows

Block	Content
Verdict banner	which plan leads on pass rate (or a tie / no-execution variant)
Report context	test project, Plan A, Plan B, build, platform
Plan metrics (×2)	assigned, executed, passed, failed, blocked, not run, pass rate, defect references
Comparison table	11 metrics with Plan A, Plan B, Δ (B - A) and the better plan
Results per build	executed / passed / failed / blocked of both plans for every build
Results per platform	the same for every platform linked to the plans
CSV export	the three sections above, UTF-8 BOM, ; separator

Semantics

• The unit is the test-case version linked to the plan (testplan_tcversions); a version linked on two platforms is counted once when no platform filter is set.
• The latest execution wins (MAX(executions.id) per version in the analysed scope); an execution of a version no longer linked to the plan is ignored.
• pass_rate = passed / executed, coverage = executed / assigned, not_run = assigned - executed (never negative).
• defects = distinct execution_bugs.bug_id references (a varchar id — never summed numerically), restricted to the versions still linked to the plan.
• The per-build table takes the latest execution per (version, build); the per-platform table per (version, platform) and walks the union of both plans' platforms (a platform linked only to plan B still appears, zeros on A).
• Both plans must belong to the same test project; plans of two projects are refused (400 plans_not_comparable).

Filters

• Build — restricts the analysed executions.
• Platform — restricts the executions and the universe: only the versions linked on that platform are counted (the platform is part of the assignment).
• A build or platform of another project is 400 invalid_request.

Rights and errors

• testplan_metrics on the owning project of both plans (the project picker accepts mgt_view_tc / mgt_modify_tc / testplan_metrics, so a metrics-only user can use the report). The client tproject_id is only an assertion and never widens the query (404 project_mismatch). Comparing a plan with itself is refused by the BFF (400 invalid_request), not only by the UI.
• 400 invalid_request / plans_not_comparable / unknown_action, 401 not_authenticated|session_expired, 403 no_right, 404 tplan_not_found|tproject_not_found|project_mismatch, 405 method_not_allowed.
• Permission denials are logged at AUDIT level, so the Event Viewer stays free of Error/Warning rows.

Screenshots

Default comparison (EN):

Build filter applied:

Same-plan guard:

Romanian locale:

Verification

• Browser (EN + RO): default render from a ?tplan_id= deep link, build filter, platform filter, plan swap, same-plan state, single-plan project state, project switch, CSV export, rights-less account (empty picker, no project name disclosed), the anon redirect to login.php?note=expired, the Reports-menu entry, console clean, Lighthouse Accessibility / Best Practices / SEO 100.
• API: fixture values match the documented metrics; the full 400/401/403/404/405 matrix is green.
• Bugs found and fixed while testing:
  • Plan B defaulted to the same plan as Plan A and the tplan_id deep link was ignored → A = requested/current plan, B = the first different plan.
  • The delta green failed WCAG AA on striped rows (3.07) → darkened to 4.83.
  • Project switch asserted the URL project instead of the selected one (404 project_mismatch) → asserts the selection, waits for the in-flight init and ignores a late answer for a project already left.
  • Platform breakdown now walks the union of both plans (a B-only platform was dropped); defects only count versions still linked to the plan; a plan with no execution shows a warning banner instead of a false tie; CSV keeps negative numbers numeric and localizes the "better" column; same-plan refused by the BFF.
• Fixture: tmp/fixtures_1957.php. Suite: Issue #1957 in tmp/TLU_Test_Cases.md. Event Viewer: no new Error/Warning row.