Java: Extractor succeeds for `codeql test run` despite `module-info.java` not being supported by `-source` version

[Marcono1234]

Version

CodeQL CLI v2.5.0

Description

Similar to #5476

It appears when codeql test run encounters a module-info.java file, but the -source version is lower than 9 (e.g. 8 which is currently the default of the extractor), the extractor aborts but the extraction is considered successful, even if other source files had compilation errors as well.

The log.log file contains:

[2021-04-03 03:17:48] [ERROR] Exception running the extractor with arguments: --javacOptions -source 8 --strict-javac-errors --encoding UTF-8 --files module-info.java Test.java
[2021-04-03 03:17:48] [ERROR] Exception: 
                              java.lang.AssertionError: null
                              	at javac_extend.com.sun.tools.javac.util.Assert.error(Assert.java:155)
                              	at javac_extend.com.sun.tools.javac.util.Assert.checkNonNull(Assert.java:62)
                              	at javac_extend.com.sun.tools.javac.comp.Enter.moduleEnv(Enter.java:251)
                              	at javac_extend.com.sun.tools.javac.comp.Enter.visitModuleDef(Enter.java:543)
                              	at javac_extend.com.sun.tools.javac.tree.JCTree$JCModuleDecl.accept(JCTree.java:2858)
                              	at javac_extend.com.sun.tools.javac.comp.Enter.classEnter(Enter.java:286)
                              	at javac_extend.com.sun.tools.javac.comp.Enter.classEnter(Enter.java:301)
                              	at javac_extend.com.sun.tools.javac.comp.Enter.visitTopLevel(Enter.java:323)
                              	at javac_extend.com.sun.tools.javac.tree.JCTree$JCCompilationUnit.accept(JCTree.java:541)
                              	at javac_extend.com.sun.tools.javac.comp.Enter.classEnter(Enter.java:286)
                              	at javac_extend.com.sun.tools.javac.comp.Enter.classEnter(Enter.java:301)
                              	at javac_extend.com.sun.tools.javac.comp.Enter.complete(Enter.java:577)
                              	at javac_extend.com.sun.tools.javac.comp.Enter.main(Enter.java:561)
                              	at javac_extend.com.sun.tools.javac.main.JavaCompiler.enterTrees(JavaCompiler.java:1069)
                              	at com.semmle.extractor.java.JavaExtractor.runExtractor(JavaExtractor.java:283)
                              	at com.semmle.extractor.java.JavaExtractor.main(JavaExtractor.java:485)

Expected would be that when a module-info.java file is encountered, but the source Java version does not support it, the extraction should fail.

Reproduction steps

1. Create a module-info.java file:

   module test {
   }

2. Create a test query file Test.ql:

   import java
   
   from CompilationUnit c
   where c.fromSource()
   select c

3. Create an empty Test.expected file
4. Run codeql test run --keep-databases
   ❌ The test passes

[aeisenberg]

@github/codeql-java this seems like a bug, can someone on the team respond?