Potential dependency of the coverage tools on the platform it's run on

[chloestefantsova]

I noticed some surprising behavior of the coverage tool while working on https://dart-review.googlesource.com/c/sdk/+/438660. It happened on Patchset 6.

• I ran the tool (dart --enable-asserts pkg/front_end/test/coverage_suite.dart --tasks=5 --add-and-remove-comments --update-expectations) on a Mac. It provided some updates. I uploaded the coverage updates as Patchset 6.
• The CI reported failures on coverage tests.
• I re-ran the tool on the Mac again. It made no further changes to the code, so there was no way to update the CL on that machine to make the failed tests pass on the CI.
• I patched the CL on a Linux machine and ran the tool there. It made some updates. I uploaded the updates as Patchset 7.
• The coverage tests on the CI passed this time.

So I suspect there's some dependency of the coverage tool on the platform it's run on.

[chloestefantsova]

/cc @jensjoha @johnniwinther

[jensjoha]

Interesseting. Can you reproduce it now?

What does dart point to?
Was it a new compile or an old one?

Do you have any uncommitted tests in pkg/front_end/testcases/? (on the mac?)

E.g. in https://dart-review.googlesource.com/c/sdk/+/438660/6..7 (test log) it seems that the mac run (v6) of textual_outline.dart got an error, which might also explain the missing ignore in incremental_compiler.dart. The missing ignores in outline_builder.dart could also be consistent with there being an uncommitted test.

Maybe I should add an option to strong_suite.dart and parser_suite.dart so that they (can) only run tests known by git (and do that from coverage_suite.dart).