"Attempted to insert a duplicate test item ID" when modifying tests after running them using testGoldens from golden_toolkit

[DanTup]

import 'package:golden_toolkit/golden_toolkit.dart';
import 'package:test/test.dart';

void main() {
  group('foo', () {
    testGoldens('test 1', (tester) async {});
  });
}

Then Test: Run All Tests, wait for results, then try to modify the file (eg. add a new linebreak):

[DanTup]

This looks like it's caused by @isTestGroup on the testGoldens function. We get a test group from the discovered that uses the outline, and a test with the same name from the results.

The package should be updated, but we should not crash in this case. We could use ID formats for groups/tests to avoid colliding like this.

[DanTup]

Looks like this method used to create a group, but was changed in eBay/flutter_glove_box@f1d486c.

I've filed a PR for the annotation at eBay/flutter_glove_box#140.

[moonytoes29]

@DanTup - even without the golden tests being inside the group, we get some console errors.
Perhaps the overall number of testGoldens I have in our work project is what's causing the testing status, code lens and test count to be flaky or for the most part wrong?

I've just tested my work code after commenting out all the testGolden calls. All 229 of them. And ran the tests a number of times. The count appears to pick up the tests. No errors are coming up on the console. All seems a lot better.

So I think this might be promising. :)

[DanTup]

@moonytoes29

even without the golden tests being inside the group, we get some console errors.

Sorry, the group wasn't necessary - it's just the code I started with when I triggered it. The important of groups is that the testGoldens function is annotated with @isTestGroup and that causes us to believe there is a group with the name you supplied here at analysis time. But when you run the tests, we now get a set of test results that show a test with that name. We have to merge data from both analysis and runtime to build that tree, so we end up creating two nodes with the same ID (one is the group from the analyzer data, and one the test from the results).

The fix I made in VS Code says that groups and tests always have separate IDs (eg. they are prefixed with GROUP: and TEST:) which ensures we can never create collisions like this that will cause things to fail. However, it means that (until the testGoldens function is fixed), you'll see two nodes for the tests in the tree (one from analysis, one from test results) which is ofcourse a little odd.

If you switch to the pre-release channel noted above, it will resolve the errors. Another option is to modify your source code for golden_toolkit locally to be @isTest instead of @isTestGroup, although modifying things in your pubcache is a little weird :-)

[DanTup]

@moonytoes29 a new version of golden_toolkit was published with the annotation fix. If you can upgrade to that, everything should be good now :-)

[moonytoes29]

@moonytoes29 a new version of golden_toolkit was published with the annotation fix. If you can upgrade to that, everything should be good now :-)

Wish I could. But alongside your simple fix, they've now made that latest version dependent on having SDK >=2.15. Which my project is not!.

So I'll have to do a fork to apply just your fix for now. Then look to migrate sooner rather than later.

[DanTup]

Ah, that's a shame. Although if it's just you and not a team, I'd probably just modify the file in the pub cache to have @isTest. I don't think that's likely to change anything except fix this (otherwise, if you use the pre-release extension, it at least won't stop you running tests, you'll just have some repeated nodes in the tree).