Skip to content

feat: captureDashboard screenshot helper + fix 16 Phase-1011 regressions surfaced by visual audit#58

Closed
HanSur94 wants to merge 9 commits intomainfrom
claude/strange-shockley-cad408
Closed

feat: captureDashboard screenshot helper + fix 16 Phase-1011 regressions surfaced by visual audit#58
HanSur94 wants to merge 9 commits intomainfrom
claude/strange-shockley-cad408

Conversation

@HanSur94
Copy link
Copy Markdown
Owner

Summary

Two intertwined workstreams in one branch:

  1. Add captureDashboard screenshot helper — a pure-MATLAB primitive that renders a DashboardEngine (or a single widget) to a PNG. Designed for agentic UI testing via matlab-mcp: build a dashboard → capture → Read the PNG back into the agent for visual verification. Covers MATLAB R2024a+ (exportapp, with crop-to-panel fallback for uicontrol-only widgets), MATLAB R2020a–R2023b (exportgraphics), and Octave (print + stub-axes).

  2. Fix 16 broken examples that visual audit caught. Phase 1011 (Tag-based domain model) migrated SensorSensorTag but left the widget/example layer partially updated. All caught by running each example through captureDashboard and inspecting the PNG.

What's new

libs/Dashboard/captureDashboard.m (+ test + example)

  • captureDashboard(engine, path) — full dashboard
  • captureDashboard(engine, path, 'Widget', titleOrHandle) — single widget (embedded or detached)
  • Accepts figure handles directly as well
  • Backend dispatch mirrors DashboardEngine.exportImage: exportapp → exportgraphics → Octave print+stub-axes
  • tests/test_capture_dashboard.m — 4 function-style Octave-safe tests
  • examples/03-dashboard/example_capture_dashboard.m — documents the matlab-mcp workflow

libs/SensorThreshold/SensorTag.m

  • Added default Thresholds = {} public property as a compat shim. GaugeWidget, StatusWidget, ChipBarWidget, MultiStatusWidget, and IconCardWidget still probe sensor.Thresholds as a fallback colour/range source — the property was removed in Phase 1011 but the widget probes weren't. Now the guard ~isempty(sensor.Thresholds) cleanly evaluates to false instead of raising.

libs/Dashboard/DashboardSerializer.m

  • configToWidgets now honours the SensorResolver callback for v2.0 layouts. Before: matched only source.type='sensor' (v1.x shape). After: also matches source.type='tag' with source.key (current FastSenseWidget output). Fixes example_dashboard_engine which silently reloaded dashboards with widget.Tag = [] and then failed render() with "Add at least one line before render()".

12 example scripts migrated off legacy Sensor.X/.Y setters

Since SensorTag.X and .Y are read-only dependent properties in the v2.0 model, assignment like sTag.Y(end) = 76 raised. Migrated to the constructor-inline pattern:

% Before
sTemp = SensorTag('T-401', ...);
sTemp.X = t;
sTemp.Y(end) = 76;

% After
yTemp = 70 + 4*sin(2*pi*t/3600) + randn(1,N)*0.8;
yTemp(end) = 76;
sTemp = SensorTag('T-401', 'X', t, 'Y', yTemp);

Files: example_widget_{fastsense,gauge,group,heatmap,histogram,iconcard,multistatus,scatter,sparkline,status}.m, plus example_widget_table.m and example_dashboard_all_widgets.m (replaced ResolvedViolations / countViolations callers with peak-sample / min-max summaries since those APIs no longer exist in the Tag model — MonitorTag owns threshold state now).

Why

Without captureDashboard, visual-regression checking on MATLAB dashboards was manual. With it, the agentic loop becomes: build → render headless → PNG → Read → inspect — one turn, no human in the loop.

The Phase-1011 regressions had been sitting in main without anyone noticing because the existing test suite mostly covered library internals, not end-to-end example rendering. The very first audit run caught them all.

Test plan

  • tests/test_capture_dashboard.m → 4/4 pass
  • tests/test_dashboard_toolbar_image_export.m → 4/4 pass (regression)
  • tests/test_sensortag.m, test_monitortag.m, test_compositetag.m, test_fastsense_widget_tag.m → pass
  • All 16 previously-broken examples render via captureDashboard end-to-end through matlab-mcp
  • Spot-checked rendered PNGs for visual correctness (gauge, scatter, dashboard_info, etc.)
  • Pre-existing failures in test_multistatus_widget_tag and test_icon_card_widget_tag — both reference a Threshold(...) constructor that doesn't exist in the codebase. These failures predate this branch (Phase 1011 test debt, tracked separately).

Notes for reviewers

  • SensorTag.Thresholds = {} is intentionally a shim, not a new supported API. The right long-term fix is to migrate the five dashboard widgets off sensor.Thresholds and onto explicit MonitorTag bindings — but that's a bigger refactor and out of scope here.
  • The FastSenseWidget.fromStruct path emits TagRegistry key not found warnings during reload when a SensorResolver is provided. The warning is cosmetic (the resolver fixes up obj.Tag on the next step), but could be silenced in a follow-up.
  • example_dashboard_live.m uses sTag.X(end+1) = ... streaming-append in its live-timer callback. Not migrated in this PR since the timer path needs dedicated work; captured as an outstanding item.

🤖 Generated with Claude Code

HanSur94 and others added 8 commits April 17, 2026 14:49
…-based UI inspection

- Programmatic PNG capture of DashboardEngine, single widget panel, or detached mirror
- Three-tier backend dispatch mirroring DashboardEngine.exportImage (exportapp / exportgraphics / print + stub-axes)
- Namespaced errors (invalidTarget, notRendered, widgetNotFound, unknownOption, writeFailed)
- Octave widget-only capture falls back to whole-figure with documented caveat
- Returns absolute file path for immediate Read by matlab-mcp agent

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ashboard

- testCaptureFullDashboard: PNG exists, >1000 bytes, imread-readable
- testCaptureByWidgetTitle: per-widget capture returns non-empty PNG
- testCaptureReturnsAbsolutePath: relative filename resolves via pwd
- testCaptureUnknownOptionThrows: asserts captureDashboard:unknownOption id
- Follows tests/test_dashboard_toolbar_image_export.m structure (nPassed/nFailed counters, headless Visible=off, tempname tmp files)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ing matlab-mcp screenshot workflow

- Builds a 4-widget dashboard (fastsense + 2 number + text) with inline XData/YData
- Captures full dashboard and single widget panel via captureDashboard
- Prints both absolute PNG paths to stdout for agent Read
- Inline AGENT WORKFLOW comment block documents the 5-step matlab-mcp loop
- Detached-widget capture shown as commented-out reference

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…SUMMARY + STATE)

- Commit PLAN.md + SUMMARY.md under .planning/quick/260417-kg9-.../
- Add row to STATE.md Quick Tasks Completed table (commit 63e8d34)
- Update Session Continuity timestamp

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…rtapp+crop

exportgraphics(uipanel, ...) fails on panels containing only uicontrols
(e.g. NumberWidget) with "Figure must contain graphics". End-to-end
matlab-mcp validation of the example hit this on MATLAB R2025b.

Route widget-only captures through exportapp on the whole figure, then
crop to the panel's figure-relative pixel bounds. Uses
getpixelposition(h, true) to walk nested uipanel parents (dashboard
widgets live inside a content panel, not directly in the figure) and
offsets the crop by the figure chrome height (exportapp renders ~49px
taller than get(hFig,'Position') reports for the menubar).

Verified via matlab-mcp: full + widget PNGs produced headless on
R2025b, widget crop isolates exactly the target uipanel. Regression:
test_capture_dashboard 4/4 pass, test_dashboard_toolbar_image_export
4/4 pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Update STATE.md commit column to 5b4984e (latest) and document the
post-execution fix in the SUMMARY front-matter — the initial executor
tested the Octave path (4/4 pass) but the MATLAB R2024a+ widget-panel
path surfaced as broken only during end-to-end matlab-mcp validation.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ver to v2.0 tag shape

Two library-level gaps surfaced by running the dashboard/widget example
suite through the new captureDashboard helper:

1. SensorTag: add a default `Thresholds = {}` public property as a
   compatibility shim for the pre-Phase-1011 Sensor.Thresholds API.
   Five widgets (Gauge, Status, ChipBar, MultiStatus, IconCard) still
   probe this list as a fallback colour/range source behind
   `~isempty(sensor.Thresholds)` guards. Without the property, the
   guard itself raises "Unrecognized method, property, or field
   'Thresholds'" instead of cleanly evaluating to false. The widgets
   degrade to plain rendering when the list is empty — no runtime
   regression, just stops the probe from erroring.

2. DashboardSerializer.configToWidgets: honour the optional
   SensorResolver hook for v2.0 serialized layouts. FastSenseWidget
   now writes `source.type = 'tag'` with `source.key`, but the
   resolver branch only matched the legacy `source.type = 'sensor'`
   + `source.name` shape, so reload-from-JSON silently left
   `widget.Tag = []` and render() failed with "Add at least one line
   before render()". Now accepts both shapes.

Validated end-to-end via matlab-mcp: example_dashboard_engine now
survives a full save -> load roundtrip and renders the reloaded
dashboard. Regression: test_sensortag, test_fastsense_widget_tag,
test_monitortag, test_compositetag, test_capture_dashboard, and
test_dashboard_toolbar_image_export all still pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
SensorTag exposes X and Y only as read-only dependent properties
(backward-compat getters, no setters). 12 example scripts still used the
old `sTag.X = ...; sTag.Y = ...` assignment pattern from the pre-Phase-1011
Sensor class, plus subscripted-assign variants like `sTag.Y(end-50:end) =
val`. Visual audit via captureDashboard caught every one.

Migration pattern (all files):
- Build the Y vector in a local variable, apply any tail/window mutations
  there, then pass the completed vector to SensorTag via the 'X' / 'Y'
  inline constructor keys. No behavioural change to the rendered data.

Also rewrites two API-drift code paths that the same audit uncovered:
- example_widget_table.m + example_dashboard_all_widgets.m: replace the
  now-gone `Sensor.ResolvedViolations` alarm-log builder with a peak-sample
  synthesis. Thresholds in the v2.0 Tag model live on separate MonitorTag
  objects; these demos intentionally skip the MonitorTag wiring, so the
  synthesised log keeps the TableWidget/BarChart showcase readable without
  implying a violations pipeline.
- example_widget_status.m + example_dashboard_all_widgets.m: replace
  `sensor.countViolations()` calls in the post-render fprintf blocks with
  latest-reading / min-max summaries.

Also fixes a one-off broken line in example_widget_sparkline.m that
referenced `sCpu.Y` inside its own constructor (the result of an
incomplete auto-migration).

End-to-end validation via matlab-mcp: all 16 previously-broken examples
(12 widget demos + 4 dashboard demos) now render to PNG and look correct
under visual inspection.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Performance Alert ⚠️

Possible performance regression was detected for benchmark 'FastSense Performance'.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 1.10.

Benchmark suite Current: 2517e61 Previous: e09e7fa Ratio
Instantiation mean std(1M) 2.365 ms 0.999 ms 2.37
Render mean std(1M) 3.323 ms 2.917 ms 1.14
Render mean std50M) 3.611 ms 0.608 ms 5.94
Downsample mean ( std00M) 0.85 ms 0.049 ms 17.35
Downsample mean ( std00M) 34.576 ms 0.049 ms 705.63
Instantiation mean ( std00M) 9845.746 ms 40.042 ms 245.89
Render mean ( std00M) 479.231 ms 2.905 ms 164.97
Dashboard broadcastTimeRange mean 0.227 ms 0.11 ms 2.06
Dashboard broadcastTimeRange stdmean 0.157 ms 0.025 ms 6.28

This comment was automatically generated by workflow using github-action-benchmark.

CC: @HanSur94

… quartiles

Three post-PR fixes surfaced by running CI on PR #58:

1. **Blank-line lint (MATLAB Lint)** — squeeze leftover double-blank lines
   in 8 examples where the original `sTag.X = ...; sTag.Y = ...` block had
   a trailing blank and the migration to constructor-inline data left the
   second blank in place. Now `mh_style` runs clean on all 14 edited
   example files.

2. **example_dashboard_advanced.m** — missed in the original audit pass;
   still called `sensor.ResolvedViolations`. Replaced with the same
   peak-sampling alarm-log synthesis used for example_widget_table and
   example_dashboard_all_widgets (thresholds belong to MonitorTag in the
   v2.0 Tag model). This was the last "ResolvedViolations" caller.

3. **example_widget_rawaxes.m** — was calling `quantile()` which lives in
   the Statistics Toolbox. CLAUDE.md states the project must be
   toolbox-free. Replaced with an inline `quartilesNoToolbox()` helper
   that uses `sort()` + linear interpolation to compute Q1/Median/Q3.

Verified end-to-end via matlab-mcp: both previously-failing smoke-test
examples now render. `mh_style` summary: "14 file(s) analysed, everything
seems fine" for all files this PR touches.

Expected CI impact: Example Smoke Tests 24/26 -> 26/26; MATLAB Lint noise
from this PR removed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@HanSur94
Copy link
Copy Markdown
Owner Author

Closing in favor of a narrower PR containing just the captureDashboard helper.

This PR originally bundled two things:

  1. The new captureDashboard screenshot helper for agentic UI testing
  2. Fixes to 16 examples caught by the very first visual audit using that helper

While this PR was open, PR #57 landed on main with equivalent fixes for the example regressions (commit 30927c7 "fix(examples,style): migrate examples to post-SensorTag API + clear lint") — so the second track is now redundant. This PR went dirty with merge conflicts as a result.

I'll reopen with a clean branch containing only the captureDashboard helper + its test + its example. That's the unique value that isn't already on main.

@HanSur94 HanSur94 closed this Apr 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant