Add documentation, use subcommands and support multiple import files.

substratevm/CONFIGURE.md

@@ -0,0 +1,41 @@
+# Assisted Configuration of Native Image Builds
+
+_NOTE: the described agent is currently only supported on Linux._
+
+Native images are built ahead of runtime and their build relies on a static analysis of which code will be reachable. However, this analysis cannot always completely predict all usages of the Java Native Interface (JNI), Java reflection, dynamic proxy objects (`java.lang.reflect.Proxy`) or class-path resources (`Class.getResource`). Undetected usages of these dynamic features need to be provided to the `native-image` tool in the form of configuration files.
+
+In order to make preparing these configuration files easier and more convenient, GraalVM provides an _agent_ that traces all usages of dynamic features of an execution on a regular Java VM. It can be enabled on the command line of the GraalVM `java` command:
+
+```
+/path/to/graalvm/bin/java -agentlib:native-image-agent=output=/path/to/trace-file.json ...
+```
+
+During execution, the agent interfaces with the Java VM to intercept all calls that look up classes, methods, fields, resources or request proxy accesses and writes a trace file with the specified path.
+
+Next, the `native-image-configure` tool can transform the trace file to configuration files for native image builds. This tool itself must first be built with the command `native-image --tool:native-image-configure`. Then, the tool can be used as follows:
+
+```
+native-image-configure process-trace                          \
+                       --jni-output jni-config.json           \
+                       --reflect-output reflect-config.json   \
+                       --proxy-output proxy-config.json       \
+                       --resources-output resource-args.txt   \
+                       /path/to/trace-file.json
+```
+
+This invocation reads and processes `trace-file.json` and generates the files `jni-config.json`, `reflect-config.json`, `proxy-config.json` and `resource-params.txt` (but not all these arguments must always be specified, see `native-image-configure help`). The `.json` files are stand-alone configuration files, while `resource-args.txt` contains command-line arguments to `native-image`.
+
+It is advisable to manually review the generated configuration files. Because the agent observes only a single execution, the resulting configurations can be missing elements that are used in code paths that were not executed. It could also make sense to simplify the generated configurations to make any future manual maintenance easier.
+
+Failed lookups of classes, methods, fields or resources are recorded in the trace, but are not included in the generated configurations by default. Likewise, accesses from inside the Java class library or the Java VM (such as in `java.nio`) are filtered by default. For testing purposes, filtering can be disabled by passing `--no-filter` to `native-image-configure`, but the generated configuration files are likely unsuitable for a build.
+
+Finally, the generated configuration files can be used in a `native-image` build as follows:
+
+```
+native-image -H:JNIConfigurationFiles=jni-config.json                  \
+             -H:ReflectionConfigurationFiles=reflect-config.json       \
+             -H:DynamicProxyConfigurationFiles=proxy-config.json       \
+             -H:IncludeResources=some/resource/on/classpath.txt        \
+             -H:IncludeResources=some/other/resource/on/classpath.txt  \
+             ...
+```

substratevm/DYNAMIC_PROXY.md

@@ -1,10 +1,11 @@
 # Dynamic proxies on Substrate VM
+(See also the [guide on assisted configuration of Java dynamic proxies and other dynamic features](CONFIGURE.md))
 
 Java dynamic proxies, implemented by `java.lang.reflect.Proxy`, provide a mechanism which enables object level access control by routing all method invocations through a `java.lang.reflect.InvocationHandler`.
 Dynamic proxy classes are generated from a list of interfaces.
 
 Substrate VM doesn't provide machinery for generating and interpreting bytecodes at run time.
-Therefore all dynamic proxy classes need to be generated at native image build time
+Therefore all dynamic proxy classes need to be generated at native image build time.
 
 
 # Automatic detection

substratevm/JNI.md

@@ -1,4 +1,5 @@
 # Java Native Interface (JNI) on Substrate VM
+(See also the [guide on assisted configuration of JNI and other dynamic features](CONFIGURE.md))
 
 JNI is a native API that enables Java code to interact with native code and vice versa.
 This page gives an overview over the JNI implementation in Substrate VM.

substratevm/REFLECTION.md

@@ -1,4 +1,5 @@
 # Reflection on Substrate VM
+(See also the [guide on assisted configuration of Java reflection and other dynamic features](CONFIGURE.md))
 
 Java reflection support (the `java.lang.reflect.*` API) enables Java code to examine its own classes, methods and fields and their properties at runtime.
 

substratevm/RESOURCES.md

@@ -1,4 +1,5 @@
 # Accessing resources in Substrate VM images
+(See also the [guide on assisted configuration of Java resources and other dynamic features](CONFIGURE.md))
 
 By default, the native-image builder will not integrate any of the resources which are on the classpath during image building into the image it creates.
 To make calls such as `Class.getResource()`, `Class.getResourceAsStream()` (or the corresponding ClassLoader methods) return specific resources (instead of null), the resources that should be accessible at image runtime need to be explicitly specified. This can be done via a configuration file such as the following:

substratevm/mx.substratevm/suite.py

@@ -637,6 +637,7 @@
             "subDir": "src",
             "sourceDirs": [
                 "src",
+                "resources",
             ],
             "dependencies": [
                 "com.oracle.svm.core",

substratevm/src/com.oracle.svm.configure/resources/Help.txt

@@ -0,0 +1,35 @@
+
+GraalVM native-image-configure tool
+
+This tool can be used to prepare a configuration of JNI, reflection and
+resources for a native-image build.
+
+Usage: native-image-configure process-trace [options] tracefile...
+       native-image-configure help
+
+process-trace             processes trace file(s) from the agent.
+    --reflect-output <path>
+                          write a reflection configuration file with the given
+                          path. This file can be later provided to native-image
+                          with -H:ReflectionConfigurationFiles=<path>.
+    --jni-output <path>
+                          write a JNI configuration file with the given path.
+                          This file can be later provided to native-image with
+                          -H:JNIConfigurationFiles=<path>.
+    --proxy-output <path>
+                          write a dynamic proxy configuration file at the given
+                          path. This file can be later provided to native-image
+                          with -H:DynamicProxyConfigurationFiles=<path>.
+    --resources-output <path>
+                          write a file with used resources (getResource) to the
+                          given path. Each line is an option to native-image
+                          for including a specific resource. The paths might
+                          need to be adjusted to the build directory structure.
+    --no-filter
+                          usage of JNI, reflection and resources in internal
+                          classes does not have to be configured for builds
+                          with native-image and by default, is excluded from
+                          the generated configurations. This option disables
+                          this filter.
+
+help                      prints this help message.

substratevm/src/com.oracle.svm.configure/src/com/oracle/svm/configure/ConfigurationTool.java

@@ -24,83 +24,104 @@
  */
 package com.oracle.svm.configure;
 
+import java.io.BufferedReader;
 import java.io.IOException;
+import java.io.InputStream;
+import java.io.InputStreamReader;
+import java.io.Reader;
 import java.io.Writer;
 import java.nio.charset.StandardCharsets;
 import java.nio.file.Files;
 import java.nio.file.Path;
 import java.nio.file.Paths;
+import java.util.ArrayList;
 import java.util.Arrays;
 import java.util.Iterator;
+import java.util.List;
+import java.util.stream.Collectors;
 
 import com.oracle.svm.configure.json.JsonWriter;
 import com.oracle.svm.configure.trace.TraceProcessor;
+import com.oracle.svm.core.util.VMError;
 
 public class ConfigurationTool {
-    private static void printUsage() {
-        System.err.println("Usage: --trace file [--reflect-output file] [--jni-output file] [--proxy-output file] [--resources-output file] [--no-filter]");
+
+    private static final String HELP_TEXT = getResource("/Help.txt") + System.lineSeparator();
+
+    private static class UsageException extends RuntimeException {
+        static final long serialVersionUID = 1L;
+
+        UsageException(String message) {
+            super(message);
+        }
     }
 
     public static void main(String[] args) {
-        if (args.length == 0) {
-            System.err.println("No arguments provided.");
-            printUsage();
-            return;
-        }
         try {
+            if (args.length == 0) {
+                throw new UsageException("No arguments provided.");
+            }
             Iterator<String> argsIter = Arrays.asList(args).iterator();
             String first = argsIter.next();
-            if (first.equals("--trace")) {
+            if (first.equals("process-trace")) {
                 processTrace(argsIter);
+            } else if (first.equals("help") || first.equals("--help")) {
+                System.out.println(HELP_TEXT);
             } else {
-                System.err.println("Unknown argument: " + first);
-                printUsage();
+                throw new UsageException("Unknown subcommand: " + first);
             }
+        } catch (UsageException e) {
+            System.err.println(e.getMessage() + System.lineSeparator() +
+                            "Use 'native-image-configure help' for usage.");
+            System.exit(2);
         } catch (Exception e) {
             e.printStackTrace();
+            System.exit(1);
         }
     }
 
     private static void processTrace(Iterator<String> argsIter) throws IOException {
-        Path traceInputPath;
+        List<Path> traceInputPaths = new ArrayList<>();
         boolean filter = true;
         Path reflectOutputPath = null;
         Path jniOutputPath = null;
         Path proxyOutputPath = null;
         Path resourcesOutputPath = null;
-        try {
-            traceInputPath = Paths.get(argsIter.next());
-            while (argsIter.hasNext()) {
-                String current = argsIter.next();
-                switch (current) {
-                    case "--reflect-output":
-                        reflectOutputPath = Paths.get(argsIter.next());
-                        break;
-                    case "--jni-output":
-                        jniOutputPath = Paths.get(argsIter.next());
-                        break;
-                    case "--proxy-output":
-                        proxyOutputPath = Paths.get(argsIter.next());
-                        break;
-                    case "--resources-output":
-                        resourcesOutputPath = Paths.get(argsIter.next());
-                        break;
-                    case "--no-filter":
-                        filter = false;
-                        break;
-                    default:
-                        throw new IllegalArgumentException("Unknown argument: " + current);
-                }
+
+        while (argsIter.hasNext()) {
+            String current = argsIter.next();
+            switch (current) {
+                case "--reflect-output":
+                    reflectOutputPath = Paths.get(argsIter.next());
+                    break;
+                case "--jni-output":
+                    jniOutputPath = Paths.get(argsIter.next());
+                    break;
+                case "--proxy-output":
+                    proxyOutputPath = Paths.get(argsIter.next());
+                    break;
+                case "--resources-output":
+                    resourcesOutputPath = Paths.get(argsIter.next());
+                    break;
+                case "--no-filter":
+                    filter = false;
+                    break;
+                default:
+                    traceInputPaths.add(Paths.get(current));
+                    break;
             }
-        } catch (Exception e) {
-            System.err.println(e.getMessage());
-            printUsage();
-            return;
         }
 
         TraceProcessor p = new TraceProcessor();
         p.setFilterEnabled(filter);
-        p.process(Files.newBufferedReader(traceInputPath));
+        if (traceInputPaths.isEmpty()) {
+            throw new UsageException("No trace files specified.");
+        }
+        for (Path path : traceInputPaths) {
+            try (Reader reader = Files.newBufferedReader(path)) {
+                p.process(reader);
+            }
+        }
         if (reflectOutputPath != null) {
             try (JsonWriter writer = new JsonWriter(reflectOutputPath)) {
                 p.getReflectionConfiguration().printJson(writer);
@@ -122,4 +143,14 @@ private static void processTrace(Iterator<String> argsIter) throws IOException {
             }
         }
     }
+
+    private static String getResource(String resourceName) {
+        try (InputStream input = ConfigurationTool.class.getResourceAsStream(resourceName)) {
+            BufferedReader reader = new BufferedReader(new InputStreamReader(input, StandardCharsets.UTF_8));
+            return reader.lines().collect(Collectors.joining("\n"));
+        } catch (IOException e) {
+            VMError.shouldNotReachHere(e);
+        }
+        return null;
+    }
 }

substratevm/src/com.oracle.svm.configure/src/com/oracle/svm/configure/trace/TraceProcessor.java

@@ -65,6 +65,7 @@ public ResourceConfiguration getResourceConfiguration() {
 
     @SuppressWarnings("unchecked")
     public void process(Reader reader) throws IOException {
+        setInLivePhase(false);
         JSONParser parser = new JSONParser(reader);
         List<Map<String, ?>> trace = (List<Map<String, ?>>) parser.parse();
         processTrace(trace);
@@ -75,7 +76,7 @@ private void processTrace(List<Map<String, ?>> trace) {
             try {
                 processEntry(entry);
             } catch (Exception e) {
-                logWarning("Error processing log entry: " + e.getMessage() + ": " + entry.toString());
+                logWarning("Error processing log entry: " + e.toString() + ": " + entry.toString());
             }
         }
     }