Uneven sharding problem when a test file has only a few tests

[isoos]

We have (relative) expensive integration tests with with one (or sometimes two) test case per file, 8 *_test.dart files total. When using --total-shards 3, two of the indexes get 1-1 test cases, while the third index gets the rest of the tests. I would have expected a more even distribution of the tests among the shards.

Is there a workaround to make shard assignments manually?

[jakemac53]

You could tag the tests (shard1, shard2, etc) and then run tests using those tags? I don't think there is any explicit support for configuring the normal sharding.

[jakemac53]

Note that test loading is actually pretty expensive, so shards do not load the tests they aren't going to run, and thus they do not know how many tests are within a given file. All tests are just assumed to be roughly the same size.

I could imagine maybe some annotation configuration to assign a "weight" to a given test suite, which could be taken into account, but it isn't likely that we would be able to prioritize implementing that.