Expose the diff from ComparisonResult

[Piinks]

This exposes the diff percentage of a ComparisonResult, which will enable sub-classed LocalFileComparators to set a tolerance level for golden file testing.

Pre-launch Checklist

• I read the Contributor Guide and followed the process outlined there for submitting PRs.
• I read the Tree Hygiene wiki page, which explains my responsibilities.
• I read and followed the Flutter Style Guide, including Features we expect every widget to implement.
• I signed the CLA.
• I listed at least one issue that this PR fixes in the description above.
• I updated/added relevant documentation (doc comments with ///).
• I added new tests to check the change I am making or feature I am adding, or Hixie said the PR is test-exempt.
• All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

[burhanrashid52]

Can anyone help me to understand how do we set diff tolerance levels for the golden test ? I am not sure how I will use these api in my tests.
Currently I’ve golden test matcher like this.

await expectLater(find.byType(MyTestWidget),
    matchesGoldenFile('my_test_widget.png'));

How do I set a diff tolerance level for this test ?
@Piinks

[felix-barz-brickmakers]

anyone?

[CaseyHillers]

Here's an example with a comparator where you can customize the tolerance.

class GoldenDiffComparator extends LocalFileComparator {
  GoldenDiffComparator(String testFile, {double tolerance = 0.005,}) : super(Uri.parse(testFile));

  @override
  Future<bool> compare(Uint8List imageBytes, Uri golden) async {
    final ComparisonResult result = await GoldenFileComparator.compareLists(
      imageBytes,
      await getGoldenBytes(golden),
    );

    if (!result.passed && result.diffPercent > _kGoldenDiffTolerance) {
      final String error = await generateFailureOutput(result, golden, basedir);
      throw FlutterError(error);
    }
    if (!result.passed) {
      log('A tolerable difference of ${result.diffPercent * 100}% was found when comparing $golden.');
    }
    return result.passed || result.diffPercent <= _kGoldenDiffTolerance;
  }
}

Then in the test you can call it like

goldenFileComparator = GoldenDiffComparator(path.join(
  (goldenFileComparator as LocalFileComparator).basedir.toString(),
  goldenFileKey,
));

expectLater(actual, matchesGoldenFile(goldenPath), reason: reason, skip: skip || !Platform.isLinux);

[fzyzcjy]

Hi, may I know what is goldenFileKey?

[fzyzcjy]

Seems that the code above does not work for macos. The following works:

MyLocalFileComparator(
      Uri.file(path.join(path.fromUri((goldenFileComparator as LocalFileComparator).basedir), 'just_arbitrary_string.dart')));

[tp]

@Piinks When this is multiplied by 100, it looks like this is not a percentage to begin with, meaning it's 0-1 instead of 0-100, right?

Just was confused by some downstream code multiplying by 100, which I found unexpected for something called "percent" already.

As we probably can not change/rename this at this point, would you be fine with commenting the range on the property itself to make this clear? Or is my confusion unfounded?

[matiasleyba]

Hi @Piinks 👋🏻
Like @tp , I found it unexpected that we’re multiplying a value that’s already a percentage. Could you clarify this when you have a moment? Thanks in advance!