Skip to content

Releases: mavoryl/golden_matrix

Version 0.19.0

Choose a tag to compare

@Autocrab Autocrab released this 26 May 17:13
  • componentMatrixGolden — new API for small widgets at their intrinsic size. Sister function to matrixGolden/screenMatrixGolden for component-level testing (buttons, badges, chips, list tiles). Instead of rendering inside a full Scaffold and capturing the whole 375×667 viewport with the widget centered in 95% whitespace, the function keeps the MaterialApp context (so theme/fonts/icons/locale all work) but anchors the widget via Align(widthFactor: 1, heightFactor: 1) and places the RepaintBoundary directly around it. The resulting PNG is exactly widget-sized plus optional padding.

    componentMatrixGolden(
      'ShadButton',
      scenarios: [
        MatrixScenario('primary',
            builder: () => const ShadButton(child: Text('Click'))),
        MatrixScenario('destructive',
            builder: () => const ShadButton.destructive(child: Text('Delete'))),
      ],
      axes: const MatrixAxes(themes: [MatrixTheme.light, MatrixTheme.dark]),
    );

    Configuration:

    • pixelRatio: double (default 2.0) — PNG resolution in physical pixels = widget logical size × this value.
    • padding: EdgeInsets (default EdgeInsets.all(8)) — visual breathing room around the widget; pass EdgeInsets.zero for the tightest crop.

    File naming drops the device segment: goldens/<test>/<scenario>/<theme>_<locale>_<dir>_<scale>.png. The devices field of MatrixAxes is ignored in component mode because intrinsic size does not depend on device geometry.

    Limitations: widgets without an intrinsic size (e.g. plain Container() with no width/height) throw a layout error — wrap them in SizedBox(width:..., height:...). Widgets that need an Overlay ancestor (Tooltip, showDialog, Hero animations across routes) still work because we keep the full MaterialApp context, but Scaffold-positioned widgets (AppBar, FAB) belong in matrixGolden.

Version 0.18.0

Choose a tag to compare

@Autocrab Autocrab released this 25 May 17:58
  • Selective text/icon font loading. New optional textFonts and iconFonts parameters on loadAppFonts() (both default true, fully backward compatible). Enables layout-deterministic golden tests where text uses the Ahem placeholder (predictable geometry across macOS/Linux CI) while icons render with real glyphs.

    // test/flutter_test_config.dart — Ahem text + real icons
    Future<void> testExecutable(FutureOr<void> Function() testMain) async {
      await loadAppFonts(textFonts: false);
      return testMain();
    }

    Icon families are detected by a substring heuristic ('icons' / 'symbols', case-insensitive) that catches MaterialIcons, CupertinoIcons, FontAwesomeIcons, MaterialSymbolsRounded/Sharp/Outlined. Icon fonts that don't follow this convention (e.g. Phosphor) are treated as text — file an issue if your project hits this.

  • MaterialIcons SDK fallback. When MaterialIcons is not in FontManifest.json (i.e. no cupertino_icons dependency or similar), the loader now reads $FLUTTER_ROOT/bin/cache/artifacts/material_fonts/MaterialIcons-Regular.otf automatically. Closes the gap where icons rendered as empty boxes in projects without that dependency. Heads-up: if you previously relied on the missing-glyph behavior for goldens, this will change those goldens to show real icon glyphs.

  • New isIconFamily(String) helper exposed via @visibleForTesting for projects that want to mirror the same filter logic.

Version 0.17.0

Choose a tag to compare

@Autocrab Autocrab released this 22 May 12:52

Added

JUnit XML report format (MatrixReportFormat.junit). Opt-in fourth report format that produces a _report.xml alongside the existing JSON / HTML / Markdown. The XML follows the de-facto JUnit schema consumed natively by GitHub Actions, GitLab CI, CircleCI, Jenkins, Buildkite, Azure DevOps, and most CI dashboards.

Each scenario becomes a , each combination a . Failures emit with the captured error message; skipped combinations emit . XML special characters are escaped.

matrixGolden(
  'ProfileCard',
  scenarios: [...],
  reportFormats: const {
    MatrixReportFormat.html,
    MatrixReportFormat.markdown,
    MatrixReportFormat.junit, // ← opt-in
  },
);

GitHub Actions integration is one extra step:

- name: publish JUnit test results
  if: always()
  uses: dorny/test-reporter@v1
  with:
    name: Golden Matrix
    path: '**/test/golden/goldens/*_report.xml'
    reporter: java-junit

(Job needs permissions: { checks: write, contents: read }.)

Result in the UI: a "Golden Matrix" check that lists every test → scenario → combination as a tree, with failures annotated inline in PR diffs.

Monorepo / melos friendly

Multi-module setups aggregate XMLs via path glob — no special API. Recommended pattern: prefix matrixGolden test names with the module
('wallet/Button') to avoid tree collisions. MatrixGoldenRegrocess-local, so per-module flutter test invocations don'tcross-contaminate.

Version 0.16.0

Choose a tag to compare

@Autocrab Autocrab released this 22 May 11:35

Three additive items, one deprecation. No breaking changes.

Added

reportFormats: Set — per-format toggle on matrixGolden / screenMatrixGolden. Defaults to {json, html, markdown}. Pass a subset for CI-only or local-only workflows; const {} disables reports entirely.

matrixGolden(
  ...,
  reportFormats: const {MatrixReportFormat.markdown}, // CI: step summary only
);

isCiEnvironment — exported bool getter for best-effort CI detection. Recognises CI=true|1 (GitHub Actions, GitLab CI, CircleCI, Travis, Buildkite, Drone, Netlify) plus vendor env-vars for Azure Pipelines, Bitbucket Pipelines, Codemagic, Jenkins, TeamCity, Bamboo. Lets users branch reportFormats on environment:

reportFormats: isCiEnvironment
    ? const {MatrixReportFormat.markdown}
    : const {MatrixReportFormat.html},

Cross-test orphan-subdir detection. Each matrixGolden / screenMatrixGolden call auto-registers in a process-global MatrixGoldenRegistry. New reportOrphanGoldenSubdirs({String? goldensRoot, bool fail = false}) walks the goldens root and lists top-level subdirs no test touched — catches whole renamed/deleted tests that per-test stale detection can't see. Opt in via one line in flutter_test_config.dart:

Future<void> testExecutable(FutureOr<void> Function() testMain) async {
  await loadAppFonts();
  await testMain();
  await reportOrphanGoldenSubdirs(fail: isCiEnvironment);
}

Deprecated

report: bool parameter on matrixGolden / screenMatrixGolden / runMatrixTests. Use reportFormats instead. report: true resolves to all three formats; report: false to an empty set. When both are passed, report: wins for backward compatibility. Existing code is unaffected — just an analyzer info-level warning.

Version 0.15.0

Choose a tag to compare

@Autocrab Autocrab released this 20 May 09:39

Two additive reporting upgrades plus a long-standing failure-tracking bug fix. No breaking changes.

Added

Diff thumbnails in HTML report. Every failed test now shows a 4-tile inline grid (expected · actual · diff · masked) next to the error message, pulling Flutter's own failures/_{masterImage,testImage,isolatedDiff,maskedDiff}.png outputs. No image processing on our side — we just reference what Flutter already writes. Missing tiles hide via onerror.

Markdown summary sidecar. A new _report.md is written alongside the existing JSON and HTML reports — summary list, optional ## Failed table, optional ## Stale goldens list, link to the HTML. Drop-in for GitHub Actions step summary ($GITHUB_STEP_SUMMARY), PR comment bots, Slack.

Fixed

Failed-result tracking. Golden mismatches were being recorded as status: passed in JSON / HTML / Markdown reports even though flutter test correctly marked the run as failed. Pixel-mismatch errors from the comparator are routed through FlutterError.reportError (via runAsync) instead of propagating through the matcher's await chain — so await expectLater(...) returned cleanly and we mistakenly recorded a pass. The runner now also consults tester.binding.takeException() after the matcher returns. Existing passing tests are unaffected.

Version 0.14.0

Choose a tag to compare

@Autocrab Autocrab released this 19 May 19:06

Pure additive release. Existing goldens unaffected.

Added

Automatic stale-golden detection. After every matrixGolden / screenMatrixGolden run, the runner walks the test's golden subdirectory on disk and reports any *.png files that no combination produced — orphans from renamed scenarios, dropped axes, or removed locale/device coverage.

Surfaced in three places:

  • Console summary: Stale count + full list under the existing test summary.
  • JSON report: new top-level staleGoldens field (only present when non-empty).
  • HTML report: Stale stat card + collapsible orange section listing each orphan path.

matrixGolden: SampleButton
48 total | 48 passed | 1 stale (1.0s)
Stale (orphan goldens — not produced by any combination):
- goldens/samplebutton/old_scenario/light_en_ltr_1x_phonesmall.png

On by default. Opt-out per test with detectStaleGoldens: false. Auto-skipped when fileNameBuilder is supplied. Flutter's own failures/ diff images are correctly excluded. The runner never deletes anything — you decide what to do with the list.

Version 0.13.0

Choose a tag to compare

@Autocrab Autocrab released this 18 May 11:35

golden_matrix 0.13.0 — BREAKING (only if you used tolerance:)

Fixed

_TolerantComparator was looking up goldens in the wrong directory.

LocalFileComparator(Uri testFile) expects a test-file URI and applies dirname() to derive basedir. We passed delegate.basedir directly, which shifted the effective basedir up one level. Tolerance-enabled goldens were silently generated and matched at a wrong path.

Migration

For every matrixGolden / screenMatrixGolden call using tolerance::

  • Move existing goldens down one directory to the corrected path, or
  • Run flutter test --update-goldens once to regenerate (pixel content unchanged, only path).

Other

  • Example suite now applies tolerance: 0.01% everywhere (with 0.02% for TightRow_OverflowDemo) to absorb cross-macOS anti-aliasing noise.
  • GitHub Actions CI workflow added: analyze + unit tests on ubuntu-latest, example pixel goldens on macos-latest.

Version 0.12.0

Choose a tag to compare

@Autocrab Autocrab released this 18 May 11:05

golden_matrix 0.12.0 — Post-pump-state release

Three orthogonal parameters that together unlock a huge class of previously-impossible tests. Symmetric across matrixGolden and screenMatrixGolden.

Added

setup — interact before capture

matrixGolden(
  'LoginForm',
  scenarios: [...],
  setup: (tester, c) async {
    await tester.tap(submitButton);
    await tester.pumpAndSettle();
  },
);

Runs after pumpAndSettle, before the golden is captured. Tap, scroll, enter text — snapshot the post-interaction state.

freezeAnimations — kill infinite shimmer/skeletons

matrixGolden(
  'UserCardSkeleton',
  scenarios: [...],
  freezeAnimations: true,
);

Wraps the tree in TickerMode(enabled: false). Halts every AnimationController / Ticker — shimmer, skeletons, Lottie, loaders. No more pumpAndSettle hangs.

captureAfter — snapshot a specific frame

matrixGolden(
  'SlideInDialog',
  scenarios: [...],
  captureAfter: const Duration(milliseconds: 150),
);

Switches the runner into deterministic-frame mode: pump(duration) instead of pumpAndSettle. Lock to a specific mid-animation frame.

Pure additive — existing goldens unaffected. 168 internal tests + 144 pixel goldens pass without --update-goldens.

Version 0.11.0

Choose a tag to compare

@Autocrab Autocrab released this 18 May 06:52

Pure additive release — existing goldens unaffected.

Added

wrapApp — app-level decorator for matrixGolden
New optional parameter that wraps the auto-built MaterialApp from the outside — the seam for DI providers that live above MaterialApp:

matrixGolden(
  'ProfileCard',
  scenarios: [...],
  axes: axes,
  wrapApp: (app, combination) => ProviderScope(
    overrides: [userRepoProvider.overrideWithValue(FakeUserRepo())],
    child: app,
  ),
);

Works with ProviderScope (Riverpod), BlocProvider / MultiBlocProvider, MultiProvider, or any root-level InheritedWidget. The callback receives the current MatrixCombination so providers can vary per scenario.

Complementary to existing wrapChild (which sits inside MaterialApp.home for layout shells).

Other

  • Added pub.dev topics: testing, golden-tests, widget-testing, visual-regression, snapshot-testing.

Version 0.10.0

Choose a tag to compare

@Autocrab Autocrab released this 18 May 05:48

Pure additive release — no breaking changes.

Added

Device presets

  • Modern iPhones: iphone15Pro, iphone16ProMax
  • Modern Android: pixel8, pixel8Pro, galaxyS24
  • Foldables: galaxyZFoldFolded, galaxyZFoldUnfolded
  • iPads: ipadMini, ipadAir, ipadPro11, ipadPro11Landscape, ipadPro13, ipadPro13Landscape

copyWith() on MatrixAxes, MatrixDevice, MatrixCombination — extend a preset, rotate a device, or fabricate a near-identical combination without re-declaring every field.

final axes = MatrixPreset.componentFull.axes.copyWith(
  devices: [...MatrixPreset.componentFull.axes.devices, MatrixDevice.ipadPro13],
);