Multitest generates tests into single folder, even if tests have same names #27139
Labels
area-infrastructure
Use area-infrastructure for SDK infrastructure issues, like continuous integration bot changes.
P1
A high priority bug; for example, a single project is unusable or has many test failures
type-bug
Incorrect behavior (everything from a crash to more subtle misbehavior)
Please, make sure you are familiar with https://github.com/dart-lang/sdk/blob/master/tools/testing/dart/multitest.dart
The problem is dicovered with co19 test suite.
Let's imagine we have two tests in the suite:
col9\Language\Expressions\Logical_Boolean_Expressions\syntax_t10.dart
and
col9\Language\Expressions\Shift\syntax_t10.dart
The tests have the same file name, but they reside in different folders.
The tests have different expected results: dynamic type error and runtime error
So, when test suite is executed, there is wrong test result:
Executing the test manually shows that test result is wrong.
The investigation shows, that test files are generated to the same folder out/platform/generated_tests/co19. So there is only one set of generated tests
syntax_t10_none.dart
syntax_t10_01.dart
with the content, that corresponds tocol9\Language\Expressions\Shift\syntax_t10.dart test
. If I make the test case keys unique,then three files are generated
syntax_t10_none.dart
syntax_t10_key1.dart
syntax_t10_key2.dart
, which is not good either. (Although this fixes the wrong test result problem).This problem blocks fixing dart-lang/co19#69 because we have modified a lot of tests and are not able to execute the test suite before committing.
The text was updated successfully, but these errors were encountered: