Repository navigation
Releases: mavoryl/golden_matrix
Release list
Version 0.19.0
-
componentMatrixGolden— new API for small widgets at their intrinsic size. Sister function tomatrixGolden/screenMatrixGoldenfor component-level testing (buttons, badges, chips, list tiles). Instead of rendering inside a fullScaffoldand capturing the whole 375×667 viewport with the widget centered in 95% whitespace, the function keeps theMaterialAppcontext (so theme/fonts/icons/locale all work) but anchors the widget viaAlign(widthFactor: 1, heightFactor: 1)and places theRepaintBoundarydirectly around it. The resulting PNG is exactly widget-sized plus optional padding.componentMatrixGolden( 'ShadButton', scenarios: [ MatrixScenario('primary', builder: () => const ShadButton(child: Text('Click'))), MatrixScenario('destructive', builder: () => const ShadButton.destructive(child: Text('Delete'))), ], axes: const MatrixAxes(themes: [MatrixTheme.light, MatrixTheme.dark]), );
Configuration:
pixelRatio: double(default2.0) — PNG resolution in physical pixels = widget logical size × this value.padding: EdgeInsets(defaultEdgeInsets.all(8)) — visual breathing room around the widget; passEdgeInsets.zerofor the tightest crop.
File naming drops the device segment:
goldens/<test>/<scenario>/<theme>_<locale>_<dir>_<scale>.png. Thedevicesfield ofMatrixAxesis ignored in component mode because intrinsic size does not depend on device geometry.Limitations: widgets without an intrinsic size (e.g. plain
Container()with nowidth/height) throw a layout error — wrap them inSizedBox(width:..., height:...). Widgets that need anOverlayancestor (Tooltip,showDialog,Heroanimations across routes) still work because we keep the fullMaterialAppcontext, butScaffold-positioned widgets (AppBar, FAB) belong inmatrixGolden.
Version 0.18.0
-
Selective text/icon font loading. New optional
textFontsandiconFontsparameters onloadAppFonts()(both defaulttrue, fully backward compatible). Enables layout-deterministic golden tests where text uses the Ahem placeholder (predictable geometry across macOS/Linux CI) while icons render with real glyphs.// test/flutter_test_config.dart — Ahem text + real icons Future<void> testExecutable(FutureOr<void> Function() testMain) async { await loadAppFonts(textFonts: false); return testMain(); }
Icon families are detected by a substring heuristic (
'icons'/'symbols', case-insensitive) that catchesMaterialIcons,CupertinoIcons,FontAwesomeIcons,MaterialSymbolsRounded/Sharp/Outlined. Icon fonts that don't follow this convention (e.g.Phosphor) are treated as text — file an issue if your project hits this. -
MaterialIcons SDK fallback. When
MaterialIconsis not inFontManifest.json(i.e. nocupertino_iconsdependency or similar), the loader now reads$FLUTTER_ROOT/bin/cache/artifacts/material_fonts/MaterialIcons-Regular.otfautomatically. Closes the gap where icons rendered as empty boxes in projects without that dependency. Heads-up: if you previously relied on the missing-glyph behavior for goldens, this will change those goldens to show real icon glyphs. -
New
isIconFamily(String)helper exposed via@visibleForTestingfor projects that want to mirror the same filter logic.
Version 0.17.0
Added
JUnit XML report format (MatrixReportFormat.junit). Opt-in fourth report format that produces a _report.xml alongside the existing JSON / HTML / Markdown. The XML follows the de-facto JUnit schema consumed natively by GitHub Actions, GitLab CI, CircleCI, Jenkins, Buildkite, Azure DevOps, and most CI dashboards.
Each scenario becomes a , each combination a . Failures emit with the captured error message; skipped combinations emit . XML special characters are escaped.
matrixGolden(
'ProfileCard',
scenarios: [...],
reportFormats: const {
MatrixReportFormat.html,
MatrixReportFormat.markdown,
MatrixReportFormat.junit, // ← opt-in
},
);
GitHub Actions integration is one extra step:
- name: publish JUnit test results
if: always()
uses: dorny/test-reporter@v1
with:
name: Golden Matrix
path: '**/test/golden/goldens/*_report.xml'
reporter: java-junit
(Job needs permissions: { checks: write, contents: read }.)
Result in the UI: a "Golden Matrix" check that lists every test → scenario → combination as a tree, with failures annotated inline in PR diffs.
Monorepo / melos friendly
Multi-module setups aggregate XMLs via path glob — no special API. Recommended pattern: prefix matrixGolden test names with the module
('wallet/Button') to avoid tree collisions. MatrixGoldenRegrocess-local, so per-module flutter test invocations don'tcross-contaminate.
Version 0.16.0
Three additive items, one deprecation. No breaking changes.
Added
reportFormats: Set — per-format toggle on matrixGolden / screenMatrixGolden. Defaults to {json, html, markdown}. Pass a subset for CI-only or local-only workflows; const {} disables reports entirely.
matrixGolden(
...,
reportFormats: const {MatrixReportFormat.markdown}, // CI: step summary only
);
isCiEnvironment — exported bool getter for best-effort CI detection. Recognises CI=true|1 (GitHub Actions, GitLab CI, CircleCI, Travis, Buildkite, Drone, Netlify) plus vendor env-vars for Azure Pipelines, Bitbucket Pipelines, Codemagic, Jenkins, TeamCity, Bamboo. Lets users branch reportFormats on environment:
reportFormats: isCiEnvironment
? const {MatrixReportFormat.markdown}
: const {MatrixReportFormat.html},
Cross-test orphan-subdir detection. Each matrixGolden / screenMatrixGolden call auto-registers in a process-global MatrixGoldenRegistry. New reportOrphanGoldenSubdirs({String? goldensRoot, bool fail = false}) walks the goldens root and lists top-level subdirs no test touched — catches whole renamed/deleted tests that per-test stale detection can't see. Opt in via one line in flutter_test_config.dart:
Future<void> testExecutable(FutureOr<void> Function() testMain) async {
await loadAppFonts();
await testMain();
await reportOrphanGoldenSubdirs(fail: isCiEnvironment);
}
Deprecated
report: bool parameter on matrixGolden / screenMatrixGolden / runMatrixTests. Use reportFormats instead. report: true resolves to all three formats; report: false to an empty set. When both are passed, report: wins for backward compatibility. Existing code is unaffected — just an analyzer info-level warning.
Version 0.15.0
Two additive reporting upgrades plus a long-standing failure-tracking bug fix. No breaking changes.
Added
Diff thumbnails in HTML report. Every failed test now shows a 4-tile inline grid (expected · actual · diff · masked) next to the error message, pulling Flutter's own failures/_{masterImage,testImage,isolatedDiff,maskedDiff}.png outputs. No image processing on our side — we just reference what Flutter already writes. Missing tiles hide via onerror.
Markdown summary sidecar. A new _report.md is written alongside the existing JSON and HTML reports — summary list, optional ## Failed table, optional ## Stale goldens list, link to the HTML. Drop-in for GitHub Actions step summary ($GITHUB_STEP_SUMMARY), PR comment bots, Slack.
Fixed
Failed-result tracking. Golden mismatches were being recorded as status: passed in JSON / HTML / Markdown reports even though flutter test correctly marked the run as failed. Pixel-mismatch errors from the comparator are routed through FlutterError.reportError (via runAsync) instead of propagating through the matcher's await chain — so await expectLater(...) returned cleanly and we mistakenly recorded a pass. The runner now also consults tester.binding.takeException() after the matcher returns. Existing passing tests are unaffected.
Version 0.14.0
Pure additive release. Existing goldens unaffected.
Added
Automatic stale-golden detection. After every matrixGolden / screenMatrixGolden run, the runner walks the test's golden subdirectory on disk and reports any *.png files that no combination produced — orphans from renamed scenarios, dropped axes, or removed locale/device coverage.
Surfaced in three places:
- Console summary: Stale count + full list under the existing test summary.
- JSON report: new top-level staleGoldens field (only present when non-empty).
- HTML report: Stale stat card + collapsible orange section listing each orphan path.
matrixGolden: SampleButton
48 total | 48 passed | 1 stale (1.0s)
Stale (orphan goldens — not produced by any combination):
- goldens/samplebutton/old_scenario/light_en_ltr_1x_phonesmall.png
On by default. Opt-out per test with detectStaleGoldens: false. Auto-skipped when fileNameBuilder is supplied. Flutter's own failures/ diff images are correctly excluded. The runner never deletes anything — you decide what to do with the list.
Version 0.13.0
golden_matrix 0.13.0 — BREAKING (only if you used tolerance:)
Fixed
_TolerantComparator was looking up goldens in the wrong directory.
LocalFileComparator(Uri testFile) expects a test-file URI and applies dirname() to derive basedir. We passed delegate.basedir directly, which shifted the effective basedir up one level. Tolerance-enabled goldens were silently generated and matched at a wrong path.
Migration
For every matrixGolden / screenMatrixGolden call using tolerance::
- Move existing goldens down one directory to the corrected path, or
- Run
flutter test --update-goldensonce to regenerate (pixel content unchanged, only path).
Other
- Example suite now applies tolerance: 0.01% everywhere (with 0.02% for TightRow_OverflowDemo) to absorb cross-macOS anti-aliasing noise.
- GitHub Actions CI workflow added: analyze + unit tests on ubuntu-latest, example pixel goldens on macos-latest.
Version 0.12.0
golden_matrix 0.12.0 — Post-pump-state release
Three orthogonal parameters that together unlock a huge class of previously-impossible tests. Symmetric across matrixGolden and screenMatrixGolden.
Added
setup — interact before capture
matrixGolden(
'LoginForm',
scenarios: [...],
setup: (tester, c) async {
await tester.tap(submitButton);
await tester.pumpAndSettle();
},
);
Runs after pumpAndSettle, before the golden is captured. Tap, scroll, enter text — snapshot the post-interaction state.
freezeAnimations — kill infinite shimmer/skeletons
matrixGolden(
'UserCardSkeleton',
scenarios: [...],
freezeAnimations: true,
);
Wraps the tree in TickerMode(enabled: false). Halts every AnimationController / Ticker — shimmer, skeletons, Lottie, loaders. No more pumpAndSettle hangs.
captureAfter — snapshot a specific frame
matrixGolden(
'SlideInDialog',
scenarios: [...],
captureAfter: const Duration(milliseconds: 150),
);
Switches the runner into deterministic-frame mode: pump(duration) instead of pumpAndSettle. Lock to a specific mid-animation frame.
Pure additive — existing goldens unaffected. 168 internal tests + 144 pixel goldens pass without --update-goldens.
Version 0.11.0
Pure additive release — existing goldens unaffected.
Added
wrapApp — app-level decorator for matrixGolden
New optional parameter that wraps the auto-built MaterialApp from the outside — the seam for DI providers that live above MaterialApp:
matrixGolden(
'ProfileCard',
scenarios: [...],
axes: axes,
wrapApp: (app, combination) => ProviderScope(
overrides: [userRepoProvider.overrideWithValue(FakeUserRepo())],
child: app,
),
);
Works with ProviderScope (Riverpod), BlocProvider / MultiBlocProvider, MultiProvider, or any root-level InheritedWidget. The callback receives the current MatrixCombination so providers can vary per scenario.
Complementary to existing wrapChild (which sits inside MaterialApp.home for layout shells).
Other
- Added pub.dev topics: testing, golden-tests, widget-testing, visual-regression, snapshot-testing.
Version 0.10.0
Pure additive release — no breaking changes.
Added
Device presets
- Modern iPhones: iphone15Pro, iphone16ProMax
- Modern Android: pixel8, pixel8Pro, galaxyS24
- Foldables: galaxyZFoldFolded, galaxyZFoldUnfolded
- iPads: ipadMini, ipadAir, ipadPro11, ipadPro11Landscape, ipadPro13, ipadPro13Landscape
copyWith() on MatrixAxes, MatrixDevice, MatrixCombination — extend a preset, rotate a device, or fabricate a near-identical combination without re-declaring every field.
final axes = MatrixPreset.componentFull.axes.copyWith(
devices: [...MatrixPreset.componentFull.axes.devices, MatrixDevice.ipadPro13],
);