Skip to content

Build Time Scanning

Luke Hutchison edited this page Nov 17, 2020 · 38 revisions

There are several different ways to perform build-time scanning. Build-time scanning can be used to achieve faster startup times at runtime, or to support Android, which does not use the standard Java bytecode format (and does not actually even have a runtime classpath).

Contents

Build-time scanning in Maven

Option 0: Run the scan from within a unit test

The simplest way to run a build-time scan is to simply run the scan in a unit test. Typically unit tests are run only at build-time, and the test classes are never placed into the final target.

However, there are a couple of caveats:

  1. Unit tests can be disabled during build, which would mean it may be possible to build the target without running the scan.
  2. It is possible that any metadata file (containing scan metadata) that you write to the target directory from the unit test may not be incorporated into the final build, depending on the relative order of running tests vs. enumerating and assembling the final build target.

Option 1: Saving names of classes of interest

At build time, you can scan the classpath, then save the names of classes that match some criterion (or the names of other objects of interest -- methods, fields etc.). For example, you can find classes annotated with a specific annotation, and write their names out to a resource file that is stored in the built jarfile. You can then load this list at runtime without having to perform another scan, and the resulting behavior should be the same as a runtime scan as long as the classpath doesn't change between build time and runtime.

public class BuildTimeScan {
    private static final String SCAN_ROOT = "com.xyz";
    private static final String OUTPUT_FILENAME = "annotated-classes.txt";
    
    public static void main(String[] args) throws Exception {
        if (args.length != 1) {
            throw new RuntimeException("Please specify output directory");
        }
        String dir = args[0];
        File jsonFile = new File(dir, OUTPUT_FILENAME);
        try (ScanResult scanResult = new ClassGraph()
                    .acceptPackages(SCAN_ROOT)
                    .enableAllInfo()
                    .scan()) {
            try (PrintWriter writer = new PrintWriter(jsonFile)) {
                for (String className :
                        scanResult.getClassesWithAnnotation("com.xyz.MyAnnotation")
                                .getNames()) {
                    writer.println(className);
                }
            }
        }
    }
}

This code is then run from Maven (or Gradle), as shown below.

Maven configuration for build-time scanning

Once you have a build-time scanning class, put the following in your pom.xml file to actually run the scanner at build-time:

<project>
  <build>
    <plugins>
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>exec-maven-plugin</artifactId>
        <version>1.6.0</version>
        <executions>
          <execution>
            <phase>process-classes</phase>
            <goals>
              <goal>java</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <!-- The main class of your build-time scanning code -->
          <mainClass>com.xyz.BuildTimeScan</mainClass>
          <!-- Pass the build output directory as a commandline param -->
          <commandlineArgs>${project.build.outputDirectory}</commandlineArgs>
          <!-- Make resources visible to build-time scanner -->
          <addResourcesToClasspath>true</addResourcesToClasspath>
          <!-- Avoid IllegalThreadStateException if you use 
               ForkJoinPool.commonPool() -- see:
                   https://stackoverflow.com/a/25166013/3950982
                   https://github.com/classgraph/classgraph/issues/221
          -->
          <cleanupDaemonThreads>false</cleanupDaemonThreads>
        </configuration>
      </plugin>
    </plugins>
  </build>
  <dependencies>
    <dependency>
      <groupId>io.github.classgraph</groupId>
      <artifactId>classgraph</artifactId>
      <version>LATEST</version>
    </dependency>
  </dependencies>
</project>

You could also try gmavenplus-plugin:

<project>
  <build>
    <plugins>
      <plugin>
        <groupId>org.codehaus.gmavenplus</groupId>
        <artifactId>gmavenplus-plugin</artifactId>
        <version>1.8.0</version>
        <executions>
          <execution>
            <phase>generate-resources</phase>
            <goals>
              <goal>execute</goal>
            </goals>
            <configuration>
              <scripts>
                <script><![CDATA[com.xyz.BuildTimeScan.main()]]>
                </script>
              </scripts>
            </configuration>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
  <dependencies>
    <dependency>
      <groupId>io.github.classgraph</groupId>
      <artifactId>classgraph</artifactId>
      <version>LATEST</version>
    </dependency>
  </dependencies>
</project>

Option 2: Serializing ScanResult to JSON

You can scan at build time, then serialize the ScanResult to your resource directory, and read the result back in at runtime without running a full scan.

Serializing the entire ScanResult (as opposed to just the names of classes matching certain criteria) is useful when you need to preserve multiple different types of metadata from the scan.

The serialized JSON is also useful for debugging, since it contains all information obtained by the scan.

(Note that the generated JSON file is much larger than just a custom generated list of matching classes/fields/methods.)

// AT BUILD TIME:

// To serialize a ScanResult:
String jsonStr = scanResult.toJSON();
// or, to prettyprint/indent the JSON
String jsonStrIndented = scanResult.toJSON(/* indentWidth = */ 2);

// AT RUNTIME:

// To deserialize a ScanResult:
try (ScanResult scanResult = ScanResult.fromJSON(jsonStr)) {
    // ...
}

Create a build-time scanning class like the following:

public class BuildTimeScan {
    private static final String SCAN_ROOT = "com.xyz";
    private static final boolean INDENT_JSON = true;
    private static final String OUTPUT_FILENAME = "scanresult.json";

    public static void main(String[] args) throws Exception {
        if (args.length != 1) {
            throw new RuntimeException("Please specify output directory");
        }
        String dir = args[0];
        File jsonFile = new File(dir, OUTPUT_FILENAME);
        try (ScanResult scanResult = new ClassGraph()
                    .acceptPackages(SCAN_ROOT)
                    .enableAllInfo()
                    .scan()) {
            String scanResultJson = scanResult.toJSON(INDENT_JSON ? 2 : 0);
            try (PrintWriter writer = new PrintWriter(jsonFile)) {
                writer.print(scanResultJson);
            }
        }
    }
}

See Maven configuration of build-time scanning above for how to run this class at build-time.

Notes:

  1. For serialization at build time and deserialization at runtime to be useful and accurate, the classpath at buildtime should be the same as the classpath expected at runtime.
  2. The mapping between classes and ClassLoaders is not preserved by serialization to JSON, so if you try to load classes based on a serialized/deserialized ScanResult, your mileage may vary (after deserialization, ClassGraph will try to create a single unified URLClassLoader that will try to load classes from the original jar and directory paths that the classes were found in, and failing that working, the context ClassLoader will be called to try to load classes).

Including scanning code directly in pom.xml

As an alternative to the exec-maven-plugin, as shown above, you could use the gmaven-plugin to put the build-time classpath scanning code directly into your pom.xml file as a Groovy script, as shown in this example for build-time scanning with Reflections.

Build-time scanning in Gradle for Android

Build-time scanning can be performed for Android projects using the following Grade code. You will need to customize the lines following the comments in square brackets.

Gradle code

buildscript {
    repositories {
        jcenter()
    }
    dependencies {
        // [Put latest ClassGraph version number here]
        classpath 'io.github.classgraph:classgraph:4.2.12'
    }
}

afterEvaluate { project ->
    // Get some task references
    def compileTask = project.tasks.compileDebugJavaWithJavac
    def mergeTask = project.tasks.mergeDebugResources

    compileTask.doLast {
        // [Set your parameters here]
        def packageToScan = "com.example.myapplication"
        def annotationName = "com.example.myapplication.annotations.MyAnnotation"
        def scanResultFilename = "scan_result.txt"

        // Create a temporary raw resource directory for saving the scan result to
        def scanResultDir = compileTask.temporaryDir.toString() + "/res/raw/"
        new File(scanResultDir).mkdirs()
        def scanResultPath = scanResultDir + scanResultFilename

        // Get location where .class files were compiled to
        def classPackageRoot = compileTask.destinationDir

        println "Scanning " + classPackageRoot
        new io.github.classgraph.ClassGraph()
                // [Configure your ClassGraph instance here]
                .overrideClasspath(classPackageRoot)
                .acceptPackages(packageToScan)
                .enableAllInfo()
                 // .verbose()
                .scan()
                .withCloseable { scanResult ->
            // [Get the scan results you're interested in here]
            def resultList = scanResult.getClassesWithAnnotation(annotationName)
            // Write the results to the output file
            def outputFile = new File(scanResultPath)
            outputFile.withWriter { writer ->
                resultList.each { ci ->
                    // Write name of annotated class to output file
                    writer.println ci.getName()
                }
            }
            println "Wrote scan result to " + outputFile
                    + " ; size = " + outputFile.length()
        }

        // Run AAPT2 to convert the raw resource file into a .flat file
        def aapt2Path = mergeTask.buildTools.getPath(
                com.android.sdklib.BuildToolInfo.PathId.AAPT2)
        def flatFileOutputPrefix = mergeTask.outputDir.toString() + "/"
        def cmd = [aapt2Path, "compile", "-o", flatFileOutputPrefix, scanResultPath]
        println "Executing: " + cmd.join(" ")
        def exec = cmd.execute()
        exec.in.eachLine {line -> println line}
        exec.err.eachLine {line -> System.err.println "ERROR: " + line}
        exec.waitFor()
        println "Wrote compiled resource as a .flat file to " + flatFileOutputPrefix
    }
}

In case the names of properties of one of the build tasks changes, the following code snippet was useful in finding out the properties supported by the two Gradle tasks compileDebugJavaWithJavac and mergeDebugResources (substitute a task name for TASK_NAME, or leave out .TASK_NAME altogether to list all the available tasks as properties of project.tasks):

println project.tasks.TASK_NAME.properties.entrySet()*.toString()
        .sort().toString().replaceAll(", ","\n")

Android code

The scan results (consisting of a list of names of classes that have the annotation @com.example.myapplication.annotations.MyAnnotation, in this example) can be read in an Android app as follows:

String scanResultFilename = "scan_result";  // Drop the filename extension, if any

List<String> annotatedClassNames = new ArrayList<>();
try (BufferedReader reader = new BufferedReader(new InputStreamReader(
        getResources().openRawResource(
            getResources().getIdentifier(
                    scanResultFilename, "raw", getPackageName()))))) {
    for (String line; (line = reader.readLine()) != null; ) {
        annotatedClassNames.add(line);
    }
} catch (Exception e) {
    // ...
}

Build-time scanning for GraalVM

Andrea Di Cesare put together this project demonstrating how to use ClassGraph for build-time scanning with GraalVM.