Missing main under `_test.dart` file

[FMorschel]

I ran the tests in a project where I wrote the _test.dart file, but since it was a plugin, I should use test_reflective_loader. I wrote the class, but I forgot to add @reflectiveTest and the main function and tried running the tests from the terminal:

00:11 +31 -1: loading essential_lints/test\fixes\replace_with_squared_box_test.dart [E]
  Failed to load "essential_lints/test\fixes\replace_with_squared_box_test.dart":
  /C:/Users/FELIP_~1/AppData/Local/Temp/dart_test.kernel.2beb11dc/test.dart:14:42: Error: Undefined name 'main'.
        internalBootstrapVmTest(() => test.main, sendPort);
                                           ^^^^

To run this test again: C:\Users\felip_0vh5fa6\dart\bin\dart.exe test essential_lints/test\fixes\replace_with_squared_box_test.dart -p vm --plain-name "loading essential_lints/test\fixes\replace_with_squared_box_test.dart"

From inside VS Code:

Connecting to VM Service at ws://127.0.0.1:15018/jbQlmHQSBV4=/ws
Connected to the VM Service.
Failed to load "D:/dev/dart/essential_lints/essential_lints/test/fixes/replace_with_squared_box_test.dart":
/C:/Users/FELIP_~1/AppData/Local/Temp/dart_test.kernel.37e10478/test.dart:14:42: Error: Undefined name 'main'.
test.dart:14
      internalBootstrapVmTest(() => test.main, sendPort);
                                         ^^^^
package:test_core/src/runner/vm/platform.dart 290:7   VMPlatform._compileToKernel
platform.dart:290
===== asynchronous gap ===========================
package:test_core/src/runner/vm/platform.dart 260:11  VMPlatform._spawnIsolate
platform.dart:260
===== asynchronous gap ===========================
package:test_core/src/runner/vm/platform.dart 83:19   VMPlatform.load
platform.dart:83
===== asynchronous gap ===========================
package:test_core/src/runner/loader.dart 244:27       Loader.loadFile.<fn>
loader.dart:244
===== asynchronous gap ===========================
package:test_core/src/runner/load_suite.dart 104:21   new LoadSuite.<fn>.<fn>
load_suite.dart:104
✖ loading D:/dev/dart/essential_lints/essential_lints/test/fixes/replace_with_squared_box_test.dart

Exited (1).

Basically, this is trying to say I have no main. But I feel like this should be a very specific message.

[jakemac53]

Note that we don't resolve any files in this package, we only parse them (for efficiency reasons).

We could check whether there is any top level function called main just by parsing, but it won't be a perfect check, technically you can export a main from another file etc. I doubt anybody is doing that though.

[FMorschel]

That would probably work, as I also assume nobody is exporting main. Or maybe wrapping this call with a try-catch for this error. I'm not sure how easy that would be, just a guess, it may be simpler.

[jakemac53]

I believe this is actually a static compilation error so its a bit harder to catch (we are actually handling it today and just forwarding the raw error). We ideally won't have hard coded dependencies on the specific text of compilation errors, although I don't know that we would have a better option.

It would also be better to just avoid trying to compile this at all as it's wasted work, I think it would be fine if we just require a top level main in all _test.dart files, if you want to delegate to the main from another library you could just explicitly do that instead void main() => other.main();.

cc @natebosch for thoughts

[natebosch]

We ideally won't have hard coded dependencies on the specific text of compilation errors

I don't think it would be a problem to hardcode "Undefined name 'main'" as a specially treated error output.

I think it would be fine if we just require a top level main in all _test.dart files

My gut reaction was to be worried about breaking some niche use case, but the workaround is indeed very straightforward. I think I lean towards this solution.