Full dashboards area run of the checklist-test skill — all 10 items driven against a live showcase (2 opus subagents, isolated boot each). Labels: qa-run + bug.
Result: 6 PASS · 2 PARTIAL · 2 FAIL. Text-only per RUNNER.md.
Environment — framework a86db175 (PR #7304) · vendored console 09987b68 · showcase seed (10 tasks / 5 statuses / 4 priorities / 14 accounts) · isolated file DB + port per batch.
🔴 FAIL — 2 items
1. chart-type-matrix — combo widgets render as grouped bars
A widget authoring chartConfig.series = [{name:'task_count',type:'bar',yAxis:'left'},{name:'avg_progress',type:'line',yAxis:'right'}] plus two yAxis entries renders as two bar series on one shared 0–100 axis: .recharts-bar = 2, .recharts-bar-rectangle = 4, .recharts-line-curve = 0, .recharts-yAxis = 1. Reproduced 3× on independent fresh browser launches.
Repro: open /_console/apps/com.example.showcase/dashboard/showcase_chart_gallery, wait for the loader to appear then clear, settle, and count those selectors inside the "Task Count vs Avg Progress" card (combo_count_vs_progress).
Root cause (source-confirmed, two halves): objectui plugin-dashboard/src/DatasetWidget.tsx resolves the family with CHART_TYPE_MAP[widgetType] ?? 'bar' and CHART_TYPE_MAP has no combo entry; and chartConfigPresentation() in the same file deliberately does not forward series/xAxis/yAxis on the ADR-0021 dataset path ("they are DERIVED from the dataset selection"), so the authored per-series type and axis can never reach the renderer.
Not a stale-bundle artifact — verified against objectui origin/main, which is 138 commits ahead of the vendored pin and behaves identically. Tension worth a decision: widgetDispatch.ts on origin/main routes combo as a SERIES_CHART_TYPES member commenting "the chart renderer has always drawn it (it derives the base family from the series)" — but on the dataset path those series are exactly what DatasetWidget refuses to forward.
Honest caveat recorded, not ticked: kpi / gauge / solid-gauge / bullet all render value-only (pixel-identical to metric) — no dial, no arc, no bullet target band was drawn for any of them.
The other 19 variants pass with mark counts reconciled against API buckets, and compareTo renders its comparison series (with a loud prescriptive 400 when no dated window is supplied, rather than the pre-#5011 silent drop).
2. saved-report-ownership — DELETE /api/v1/reports/:id answers 500, and the mismatch leaks existence
Deleting another owner's report returns 500 REPORT_DELETE_FAILED, while deleting an id that does not exist returns 204. The 500-vs-204 split is an enumeration oracle over other users' report ids — precisely what the deny-as-404 posture exists to prevent. Reproduced 2× on two distinct reports.
Root cause: the DELETE ${dataPath}/reports/:id handler's catch goes straight to res.status(500) and never calls the file-local handleValidation(res, error) helper that maps REPORT_NOT_FOUND* → 404 — the sibling DELETE …/reports/schedules/:scheduleId handler does call it, which is why that route is correct. The service layer is already right (deleteReport() returns early for unknown ids and throws REPORT_NOT_FOUND for cross-owner, commented "others get a not-found so the delete neither fires nor reveals the report's existence") — the route discards that intent. Not stale-dist. One-line fix, plus aligning the unknown-id arm to the same status to close the side channel.
Good news on the same item: #6683's owner gate is live, and the item's revision-2 flip (expected-fail probe → positive deny assertion) is correct against today's server — cross-owner GET / run / upsert-overwrite all 404, schedule list returns empty, cross-owner unschedule 404, the owner's schedule survives, ownerId cannot be spoofed on create, anonymous 401. No cross-owner 2xx anywhere.
🟡 PARTIAL — 2 items
global-filters-rescope — everything proven except the persist-through-Studio half (blocked(environment): the showcase is installed as a read-only package so Studio Publish/Save-draft are disabled; the runner followed the UI's own prescription and created a writable package, still gated). The drive half was proven instead by authoring the exact shapes the inspector patches and confirming on reload that the re-targeted widget queries paid_on while siblings query issued_on, and the opted-out widget issues no query at all. Per-widget field mapping is exact on the wire and in the echoed SQL (region on invoice widgets vs sales_region on account widgets); dataset-sourced filter options are a real server GROUP BY, not a client top-N dedupe; client-side cosmetic filtering refuted outright.system-overview-live-counts — tiles are live (each equals its own captured query and an independent direct aggregate; Active Sessions moved 25→26 together with the direct count). Partial because on the first capture the "Events by User" bar rendered its y-axis with zero bars and no empty state — its query had returned a single {user_id:null,…} row and the renderer drops the null-keyed category. Observed once, not reproduced, so recorded as a weakness per RUNNER rule 2 rather than promoted to a fail — but it is exactly the blank-widget shape this clause guards, and a board whose only audit actor is the system would meet it.
✅ PASS — 6 items
cube-query (6/6 — count and sum reconcile bucket-for-bucket with the direct /data aggregate, emitted SQL is a single-table GROUP BY so no join fan-out; the absent arm exercised on a deliberately crippled second boot → honest 404, never a silent 200-empty) · strict-widget-rejects-stray-keys (6/6 — all 11 legacy analytics keys rejected 422 with the key echoed and the ADR-0021 prescription verbatim; after 16 rejections the read-back is byte-identical to a never-existed stub) · dataset-report-authoring (8/8 — summary/matrix/joined/tabular each verified and reconciled; matrix all 8 cells + margins; joined blocks partition 244/10 exactly) · empty-null-bucket-boundaries (6/6 — empty window repaints to designed "No rows" with all 9 queries re-fired and all KPIs dropping to 0; the live null bucket is JSON null, not a '(null)' string sentinel) · chart-first-paint (4/4 — every chart had drawn marks at first paint with no resize, refuting the resize-to-draw failure this item exists for) · drill-through-range (5/5 — a month drill carries exactly that month's half-open bounds; a matrix-cell drill ANDs both dimensions using raw values, not display labels).
Smaller observations
- The metadata door accepts a dataset binding that names nothing (200 on save and publish); reference integrity is enforced only at runtime — the widget then surfaces a visible error, which is why the negative still passes.
- The ops dashboard's
globalFilter is declared field:'status' with task status options while project-bound widgets carry no filterBindings, so it also lands on showcase_project.status and zeroes those tiles (showcase authoring gap, not a platform defect).
- The System Overview board's default "Last 7 days" filter applies to
sys_user.created_at, so "Total Users" is really users created in the last 7 days — indistinguishable from the true total only because the DB was fresh. Likewise sys_session_metrics carries no active-only filter, so the "Active Sessions" label overstates what it counts.
- The
dateRange "Custom…" picker did not open under automation — flagged for re-check against a fresh console build rather than filed.
Full
dashboardsarea run of thechecklist-testskill — all 10 items driven against a live showcase (2 opus subagents, isolated boot each). Labels:qa-run+bug.Result: 6 PASS · 2 PARTIAL · 2 FAIL. Text-only per RUNNER.md.
Environment — framework
a86db175(PR #7304) · vendored console09987b68· showcase seed (10 tasks / 5 statuses / 4 priorities / 14 accounts) · isolated file DB + port per batch.🔴 FAIL — 2 items
1.
chart-type-matrix—combowidgets render as grouped barsA widget authoring
chartConfig.series = [{name:'task_count',type:'bar',yAxis:'left'},{name:'avg_progress',type:'line',yAxis:'right'}]plus twoyAxisentries renders as two bar series on one shared 0–100 axis:.recharts-bar= 2,.recharts-bar-rectangle= 4,.recharts-line-curve= 0,.recharts-yAxis= 1. Reproduced 3× on independent fresh browser launches.Repro: open
/_console/apps/com.example.showcase/dashboard/showcase_chart_gallery, wait for the loader to appear then clear, settle, and count those selectors inside the "Task Count vs Avg Progress" card (combo_count_vs_progress).Root cause (source-confirmed, two halves): objectui
plugin-dashboard/src/DatasetWidget.tsxresolves the family withCHART_TYPE_MAP[widgetType] ?? 'bar'andCHART_TYPE_MAPhas nocomboentry; andchartConfigPresentation()in the same file deliberately does not forwardseries/xAxis/yAxison the ADR-0021 dataset path ("they are DERIVED from the dataset selection"), so the authored per-series type and axis can never reach the renderer.Not a stale-bundle artifact — verified against objectui
origin/main, which is 138 commits ahead of the vendored pin and behaves identically. Tension worth a decision:widgetDispatch.tsonorigin/mainroutescomboas aSERIES_CHART_TYPESmember commenting "the chart renderer has always drawn it (it derives the base family from the series)" — but on the dataset path those series are exactly whatDatasetWidgetrefuses to forward.Honest caveat recorded, not ticked:
kpi/gauge/solid-gauge/bulletall render value-only (pixel-identical tometric) — no dial, no arc, no bullet target band was drawn for any of them.The other 19 variants pass with mark counts reconciled against API buckets, and
compareTorenders its comparison series (with a loud prescriptive 400 when no dated window is supplied, rather than the pre-#5011 silent drop).2.
saved-report-ownership—DELETE /api/v1/reports/:idanswers 500, and the mismatch leaks existenceDeleting another owner's report returns
500 REPORT_DELETE_FAILED, while deleting an id that does not exist returns204. The 500-vs-204 split is an enumeration oracle over other users' report ids — precisely what the deny-as-404 posture exists to prevent. Reproduced 2× on two distinct reports.Root cause: the
DELETE ${dataPath}/reports/:idhandler's catch goes straight tores.status(500)and never calls the file-localhandleValidation(res, error)helper that mapsREPORT_NOT_FOUND*→ 404 — the siblingDELETE …/reports/schedules/:scheduleIdhandler does call it, which is why that route is correct. The service layer is already right (deleteReport()returns early for unknown ids and throwsREPORT_NOT_FOUNDfor cross-owner, commented "others get a not-found so the delete neither fires nor reveals the report's existence") — the route discards that intent. Not stale-dist. One-line fix, plus aligning the unknown-id arm to the same status to close the side channel.Good news on the same item: #6683's owner gate is live, and the item's revision-2 flip (expected-fail probe → positive deny assertion) is correct against today's server — cross-owner GET / run / upsert-overwrite all 404, schedule list returns empty, cross-owner unschedule 404, the owner's schedule survives,
ownerIdcannot be spoofed on create, anonymous 401. No cross-owner 2xx anywhere.🟡 PARTIAL — 2 items
global-filters-rescope— everything proven except the persist-through-Studio half (blocked(environment): the showcase is installed as a read-only package so Studio Publish/Save-draft are disabled; the runner followed the UI's own prescription and created a writable package, still gated). The drive half was proven instead by authoring the exact shapes the inspector patches and confirming on reload that the re-targeted widget queriespaid_onwhile siblings queryissued_on, and the opted-out widget issues no query at all. Per-widget field mapping is exact on the wire and in the echoed SQL (regionon invoice widgets vssales_regionon account widgets); dataset-sourced filter options are a real serverGROUP BY, not a client top-N dedupe; client-side cosmetic filtering refuted outright.system-overview-live-counts— tiles are live (each equals its own captured query and an independent direct aggregate; Active Sessions moved 25→26 together with the direct count). Partial because on the first capture the "Events by User" bar rendered its y-axis with zero bars and no empty state — its query had returned a single{user_id:null,…}row and the renderer drops the null-keyed category. Observed once, not reproduced, so recorded as a weakness per RUNNER rule 2 rather than promoted to a fail — but it is exactly the blank-widget shape this clause guards, and a board whose only audit actor is the system would meet it.✅ PASS — 6 items
cube-query(6/6 — count and sum reconcile bucket-for-bucket with the direct/dataaggregate, emitted SQL is a single-table GROUP BY so no join fan-out; the absent arm exercised on a deliberately crippled second boot → honest 404, never a silent 200-empty) ·strict-widget-rejects-stray-keys(6/6 — all 11 legacy analytics keys rejected 422 with the key echoed and the ADR-0021 prescription verbatim; after 16 rejections the read-back is byte-identical to a never-existed stub) ·dataset-report-authoring(8/8 — summary/matrix/joined/tabular each verified and reconciled; matrix all 8 cells + margins; joined blocks partition 244/10 exactly) ·empty-null-bucket-boundaries(6/6 — empty window repaints to designed "No rows" with all 9 queries re-fired and all KPIs dropping to 0; the live null bucket is JSONnull, not a'(null)'string sentinel) ·chart-first-paint(4/4 — every chart had drawn marks at first paint with no resize, refuting the resize-to-draw failure this item exists for) ·drill-through-range(5/5 — a month drill carries exactly that month's half-open bounds; a matrix-cell drill ANDs both dimensions using raw values, not display labels).Smaller observations
globalFilteris declaredfield:'status'with task status options while project-bound widgets carry nofilterBindings, so it also lands onshowcase_project.statusand zeroes those tiles (showcase authoring gap, not a platform defect).sys_user.created_at, so "Total Users" is really users created in the last 7 days — indistinguishable from the true total only because the DB was fresh. Likewisesys_session_metricscarries no active-only filter, so the "Active Sessions" label overstates what it counts.dateRange"Custom…" picker did not open under automation — flagged for re-check against a fresh console build rather than filed.