Compile tests too but don't run them

[wtwhite]

Resolves #48.

The main idea of changing -Dmaven.test.skip=true to -DskipTests gave successful compilation on an example project:

wtwhite@wtwhite-vuw-vm:~/code/jcompile$ time make jars/openjdk-11.0.16/checkstyle-10.12.3.jar.done 
--snip--
SUCCESS! - copying /target/checkstyle-10.12.3.jar  into jars/openjdk-11.0.16
openjdk-11.0.16__pid1216068
openjdk-11.0.16__pid1216068

================================================


real	0m42.738s
user	0m53.025s
sys	0m6.711s

But although that resulted in more bytecode feature metadata being generated, the output jar looks to be unchanged compared to an earlier run (23):

wtwhite@wtwhite-vuw-vm:~/code/jcompile$ ( cd runs/23/ && ls -l jars/openjdk-11.0.16/checkstyle-10.12.3* )
-rw-rw-r-- 1 wtwhite wtwhite 2076902 Nov 28 23:49 jars/openjdk-11.0.16/checkstyle-10.12.3.jar
-rw-rw-r-- 1 wtwhite wtwhite  179326 Nov 28 23:49 jars/openjdk-11.0.16/checkstyle-10.12.3.jar.bytecode-features
-rw-rw-r-- 1 wtwhite wtwhite       0 Nov 28 23:49 jars/openjdk-11.0.16/checkstyle-10.12.3.jar.done
-rw-rw-r-- 1 wtwhite wtwhite    1959 Nov 28 23:49 jars/openjdk-11.0.16/checkstyle-10.12.3.jar.generated-sources
wtwhite@wtwhite-vuw-vm:~/code/jcompile$ ls -l jars/openjdk-11.0.16/checkstyle-10.12.3*
-rw-rw-r-- 1 wtwhite wtwhite 2076902 Nov 29 09:55 jars/openjdk-11.0.16/checkstyle-10.12.3.jar
-rw-rw-r-- 1 wtwhite wtwhite 1384763 Nov 29 09:55 jars/openjdk-11.0.16/checkstyle-10.12.3.jar.bytecode-features
-rw-rw-r-- 1 wtwhite wtwhite       0 Nov 29 09:55 jars/openjdk-11.0.16/checkstyle-10.12.3.jar.done
-rw-rw-r-- 1 wtwhite wtwhite    1959 Nov 29 09:55 jars/openjdk-11.0.16/checkstyle-10.12.3.jar.generated-sources

How to get the tests into the same jar? Do we even want this?

Maybe we should create a separate .tests.jar file. Doing this via Maven is probably tricky (requiring automated messing with pom.xml, and it would be different depending on which plugin(s) were used to build the existing jar) so it would be easier to build this ourselves after running Maven.

[wtwhite]

checkstyle creates its own test jar, though it's not clear that all projects will:

wtwhite@wtwhite-vuw-vm:~/code/jcompile$ ls -ltr worktrees/pid1216871-checkstyle-10.12.3/target/
total 12096
drwxrwxrwx 4 root root    4096 Nov 29 10:01 generated-sources
drwxrwxrwx 4 root root    4096 Nov 29 10:01 maven-status
-rw-rw-rw- 1 root root    1805 Nov 29 10:01 components.xml
drwxrwxrwx 4 root root    4096 Nov 29 10:01 classes
drwxrwxrwx 3 root root    4096 Nov 29 10:01 generated-test-sources
drwxrwxrwx 4 root root    4096 Nov 29 10:01 test-classes
drwxrwxrwx 3 root root    4096 Nov 29 10:01 generated-classes
drwxrwxrwx 2 root root    4096 Nov 29 10:01 maven-archiver
-rw-rw-rw- 1 root root 2076902 Nov 29 10:01 checkstyle-10.12.3.jar
-rw-rw-rw- 1 root root 4010211 Nov 29 10:01 checkstyle-10.12.3-tests.jar
-rw-rw-rw- 1 root root 4540547 Nov 29 10:01 checkstyle-10.12.3-test-sources.jar
-rw-rw-rw- 1 root root 1713685 Nov 29 10:01 checkstyle-10.12.3-sources.jar

[wtwhite]

So does commons-lang:

wtwhite@wtwhite-vuw-vm:~/code/jcompile$ STOP_BEFORE=cp time make jars/openjdk-11.0.16/commons-lang3-3.12.0.jar.done 
--snip--
[INFO] --- jar:3.2.0:test-jar (default) @ commons-lang3 ---
[INFO] Building jar: /dataset/target/commons-lang3-3.12.0-tests.jar
--snip--
wtwhite@wtwhite-vuw-vm:~/code/jcompile$ ls -ltr worktrees/pid1217397-commons-lang3-3.12.0/target/
total 2868
drwxrwxrwx 2 root root    4096 Nov 29 10:18 antrun
drwxrwxrwx 3 root root    4096 Nov 29 10:18 apidocs
drwxrwxrwx 3 root root    4096 Nov 29 10:18 generated-sources
drwxrwxrwx 3 root root    4096 Nov 29 10:18 maven-status
drwxrwxrwx 4 root root    4096 Nov 29 10:18 classes
drwxrwxrwx 2 root root    4096 Nov 29 10:18 osgi
drwxrwxrwx 3 root root    4096 Nov 29 10:18 generated-test-sources
drwxrwxrwx 4 root root    4096 Nov 29 10:18 test-classes
drwxrwxrwx 2 root root    4096 Nov 29 10:18 maven-archiver
-rw-rw-rw- 1 root root  585988 Nov 29 10:18 commons-lang3-3.12.0.jar
-rw-rw-rw- 1 root root 1154456 Nov 29 10:18 commons-lang3-3.12.0-tests.jar
-rw-rw-rw- 1 root root  651725 Nov 29 10:18 commons-lang3-3.12.0-sources.jar
-rw-rw-rw- 1 root root  496518 Nov 29 10:18 commons-lang3-3.12.0-test-sources.jar

This looks to be produced by maven-jar-plugin, configured as so:

      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-jar-plugin</artifactId>
        <executions>
          <execution>
            <goals>
              <goal>test-jar</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <archive combine.children="append">
            <manifestEntries>
              <Automatic-Module-Name>${commons.module.name}</Automatic-Module-Name>
            </manifestEntries>
          </archive>
        </configuration>
      </plugin>

Also present in checkstyle's pom.xml with a test-jar goal:

      <!-- Ensure the manifest has all the gory details -->
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-jar-plugin</artifactId>
        <version>3.3.0</version>
        <configuration>
          <archive>
            <manifest>
              <addDefaultImplementationEntries>true</addDefaultImplementationEntries>
              <addDefaultSpecificationEntries>true</addDefaultSpecificationEntries>
            </manifest>
          </archive>
          <excludes>
            <exclude>**/Input*.*</exclude>
            <exclude>**/Expected*.*</exclude>
          </excludes>
        </configuration>
        <executions>
          <execution>
            <goals>
              <goal>test-jar</goal>
            </goals>
          </execution>
        </executions>
      </plugin>

To start with, let's just check for a file of the form target/*-tests.jar and copy it out if it's there.

[wtwhite]

Works for these two projects at least. After running:

wtwhite@wtwhite-vuw-vm:~/code/jcompile$ ls -ld `find jars`
drwxrwxr-x 3 wtwhite wtwhite    4096 Nov 29 10:35 jars
drwxrwxr-x 2 wtwhite wtwhite    4096 Nov 29 10:37 jars/openjdk-11.0.16
-rw-rw-r-- 1 wtwhite wtwhite 2076902 Nov 29 10:36 jars/openjdk-11.0.16/checkstyle-10.12.3.jar
-rw-rw-r-- 1 wtwhite wtwhite 1384763 Nov 29 10:37 jars/openjdk-11.0.16/checkstyle-10.12.3.jar.bytecode-features
-rw-rw-r-- 1 wtwhite wtwhite       0 Nov 29 10:37 jars/openjdk-11.0.16/checkstyle-10.12.3.jar.done
-rw-rw-r-- 1 wtwhite wtwhite    1959 Nov 29 10:36 jars/openjdk-11.0.16/checkstyle-10.12.3.jar.generated-sources
-rw-rw-r-- 1 wtwhite wtwhite 4010211 Nov 29 10:36 jars/openjdk-11.0.16/checkstyle-10.12.3-tests.jar
-rw-rw-r-- 1 wtwhite wtwhite  585988 Nov 29 10:35 jars/openjdk-11.0.16/commons-lang3-3.12.0.jar
-rw-rw-r-- 1 wtwhite wtwhite   81400 Nov 29 10:36 jars/openjdk-11.0.16/commons-lang3-3.12.0.jar.bytecode-features
-rw-rw-r-- 1 wtwhite wtwhite       0 Nov 29 10:36 jars/openjdk-11.0.16/commons-lang3-3.12.0.jar.done
-rw-rw-r-- 1 wtwhite wtwhite      62 Nov 29 10:35 jars/openjdk-11.0.16/commons-lang3-3.12.0.jar.generated-sources
-rw-rw-r-- 1 wtwhite wtwhite 1154456 Nov 29 10:35 jars/openjdk-11.0.16/commons-lang3-3.12.0-tests.jar

[wtwhite]

A full run (26_full_run_with_compiled_tests) took 3h44m:

wtwhite@wtwhite-vuw-vm:~/code/jcompile$ tail nohup.out 

SUCCESS! - copying /target/checkstyle-10.12.2.jar  into jars/ecj-3.32.0_openjdk-11.0.19
Found tests jar /home/wtwhite/code/jcompile/worktrees/pid1762783-checkstyle-10.12.2/target/checkstyle-10.12.2-tests.jar, copying that also.
ecj-3.32.0_openjdk-11.0.19__pid1762783
ecj-3.32.0_openjdk-11.0.19__pid1762783

================================================

13386.85user 2707.08system 3:44:11elapsed 119%CPU (0avgtext+0avgdata 2875104maxresident)k
576inputs+58467568outputs (21758major+128149732minor)pagefaults 0swaps

No more builds failed than prior to compiling tests, and a test jar was recovered for about 60% of projects:

wtwhite@wtwhite-vuw-vm:~/code/jcompile$ find jars -name '*.done'|wc -l
1344
wtwhite@wtwhite-vuw-vm:~/code/jcompile$ find jars -name '*-tests.jar'|wc -l
824

😅