Add support for Java Platform Module System

Fixes compiler error where internals are not accessible to java9 related code. Furthermore, a JPMS module is defined for the datasketches-memory, that can be used to narrow any additional compiler or runtime arguments for the JPMS.
apache · Jun 17, 2021 · 2ac0b35ebd146a2fd4487c99775b602e46712a83 · 2ac0b35
1 parent 3664b8d
commit 2ac0b35ebd146a2fd4487c99775b602e46712a83
Show file tree

Hide file tree

Showing 13 changed files with 405 additions and 67 deletions.
diff --git a/README.md b/README.md
@@ -36,13 +36,49 @@ Please visit the main [DataSketches website](https://datasketches.apache.org) fo
 
 If you are interested in making contributions to this site please see our [Community](https://datasketches.apache.org/docs/Community/) page for how to contact us.
 
+---
+
+## Java Support
+
+Datasketches memory currently supports Java 8 up to and including Java 13.
+
+In order to use the library in Java 9 and above, you must provide the following runtime arguments to the JVM:
+
+```shell
+    --add-opens java.base/java.nio=org.apache.datasketches.memory \
+    --add-opens java.base/jdk.internal.misc=org.apache.datasketches.memory \
+    --add-opens java.base/jdk.internal.ref=org.apache.datasketches.memory
+```
+
+For example, to run your local application with full compatibility for the Java module system, you might use the following command:
+```shell
+  $JAVA \
+    --module-path mods \
+    --add-opens java.base/java.nio=org.apache.datasketches.memory \
+    --add-opens java.base/jdk.internal.misc=org.apache.datasketches.memory \
+    --add-opens java.base/jdk.internal.ref=org.apache.datasketches.memory \
+    --module my.main.application.module
+```
+
+where `mods` is your module path and `my.main.application.module` is your own JPMS module:
+
+```java
+module datasketches.memory.multirelease.test {
+    requires org.apache.datasketches.memory;
+}
+```
+
+
 ---
 
 ## Build Instructions
 __NOTE:__ This component accesses resource files for testing. As a result, the directory elements of the full absolute path of the target installation directory must qualify as Java identifiers. In other words, the directory elements must not have any space characters (or non-Java identifier characters) in any of the path elements. This is required by the Oracle Java Specification in order to ensure location-independent access to resources: [See Oracle Location-Independent Access to Resources](https://docs.oracle.com/javase/8/docs/technotes/guides/lang/resources.html)
 
-### JDK8/Hotspot is required to compile
-This DataSketches component is pure Java and you must compile using JDK 8 with Hotspot.
+### JDK versions required to compile
+This DataSketches component is pure Java and requires the following JDKs to compile:
+- JDK8/Hotspot
+- JDK9/Hotspot
+- JDK11/Hotspot
 
 ### Recommended Build Tool
 This DataSketches component is structured as a Maven project and Maven is the recommended Build Tool.
@@ -69,12 +105,18 @@ This will create the following Jars:
 * datasketches-memory-X.Y.Z-test-sources.jar The test source files
 * datasketches-memory-X.Y.Z-javadoc.jar  The compressed Javadocs.
 
-### Further documentation for contributors
+### Toolchains
 
-For more information on the project configuration, the following topics are discussed in more detail:
+This project makes use of Maven toolchains to ensure that the correct Java compiler version is used when compiling source files.
 
-* [Maven configuration](docs/maven.md)
-* [Multi-release jar](docs/multi-release-jar.md)
+The reference toolchains.xml can be found in .github/workflows/.toolchains.xml, and can be copied to your local maven home
+directory e.g. `~/.m2/toolchains.xml`.
+
+Alternatively, the maven commands above can be supplemented with: `--toolchains .github/workflows/.toolchains.xml`
+
+For example, to run normal unit tests:
+
+    $ mvn clean test --toolchains .github/workflows/.toolchains.xml
 
 ### Dependencies
 
@@ -85,3 +127,15 @@ There is one run-time dependency:
 
 #### Testing
 See the pom.xml file for test dependencies.
+
+---
+
+## Further documentation for contributors
+
+For more information on the project configuration, the following topics are discussed in more detail:
+
+* [Maven configuration](docs/maven.md)
+* [Multi-release jar](docs/multi-release-jar.md)
+* [Java Platform Module System](docs/module-system.md)
+
+In order to build and contribute to this project, please read the [development setup documentation](docs/development.md).
diff --git a/datasketches-memory-java8-tests/pom.xml b/datasketches-memory-java8-tests/pom.xml
@@ -35,6 +35,11 @@
       <version>${project.parent.version}</version>
       <scope>test</scope>
     </dependency>
+    <dependency>
+      <groupId>org.testng</groupId>
+      <artifactId>testng</artifactId>
+      <version>${testng.version}</version>
+    </dependency>
   </dependencies>
 
   <artifactId>datasketches-memory-java8-tests</artifactId>
@@ -43,6 +48,7 @@
   <properties>
     <maven.install.skip>true</maven.install.skip>
     <java.version>1.8</java.version>
+    <jdk-toolchain.version>8</jdk-toolchain.version>
     <maven.compiler.source>${java.version}</maven.compiler.source>
     <maven.compiler.target>${java.version}</maven.compiler.target>
   </properties>

diff --git a/datasketches-memory-java8/pom.xml b/datasketches-memory-java8/pom.xml
@@ -34,8 +34,45 @@
   <properties>
     <maven.install.skip>true</maven.install.skip>
     <java.version>1.8</java.version>
+    <jdk-toolchain.version>8</jdk-toolchain.version>
     <maven.compiler.source>${java.version}</maven.compiler.source>
     <maven.compiler.target>${java.version}</maven.compiler.target>
   </properties>
 
+  <profiles>
+
+    <profile>
+      <id>strict</id>
+      <!-- The error prone compiler is currently disabled for compilation in JDK 9+ -->
+      <activation>
+        <jdk>1.8</jdk>
+      </activation>
+      <build>
+        <pluginManagement>
+          <plugins>
+            <plugin>
+              <groupId>org.apache.maven.plugins</groupId>
+              <artifactId>maven-compiler-plugin</artifactId>
+              <version>${maven-compiler-plugin.version}</version>
+              <dependencies>
+                <dependency>
+                  <groupId>org.codehaus.plexus</groupId>
+                  <artifactId>plexus-compiler-javac-errorprone</artifactId>
+                  <version>${plexus-compiler-javac-errorprone.version}</version>
+                </dependency>
+              </dependencies>
+              <configuration>
+                <source>${maven.compiler.source}</source>
+                <target>${maven.compiler.target}</target>
+                <compilerId>javac-with-errorprone</compilerId>
+                <forceJavacCompilerUse>true</forceJavacCompilerUse>
+              </configuration>
+            </plugin>
+          </plugins>
+        </pluginManagement>
+      </build>
+    </profile>
+
+  </profiles>
+
 </project>
diff --git a/datasketches-memory-java9-tests/pom.xml b/datasketches-memory-java9-tests/pom.xml
@@ -29,21 +29,51 @@
     </parent>
 
     <dependencies>
-<!--        <dependency>-->
-<!--            <groupId>org.apache.datasketches</groupId>-->
-<!--            <artifactId>datasketches-memory-java9</artifactId>-->
-<!--            <version>${project.version}</version>-->
-<!--        </dependency>-->
+        <dependency>
+            <groupId>org.apache.datasketches</groupId>
+            <artifactId>datasketches-memory-java9</artifactId>
+            <version>${project.version}</version>
+        </dependency>
+        <dependency>
+            <groupId>org.testng</groupId>
+            <artifactId>testng</artifactId>
+            <version>${testng.version}</version>
+        </dependency>
     </dependencies>
 
     <artifactId>datasketches-memory-java9-tests</artifactId>
     <name>${project.artifactId}</name>
 
     <properties>
         <maven.install.skip>true</maven.install.skip>
+        <maven.javadoc.skip>true</maven.javadoc.skip>
         <java.version>9</java.version>
+        <jdk-toolchain.version>9</jdk-toolchain.version>
         <maven.compiler.source>${java.version}</maven.compiler.source>
         <maven.compiler.target>${java.version}</maven.compiler.target>
     </properties>
 
+    <build>
+        <plugins>
+
+            <plugin>
+                <!-- Apache Parent pom, pluginManagement-->
+                <groupId>org.apache.maven.plugins</groupId>
+                <artifactId>maven-surefire-plugin</artifactId>
+                <version>${maven-surefire-plugin.version}</version>
+                <configuration>
+                    <argLine>
+                        --add-opens java.base/jdk.internal.ref=org.apache.datasketches.memory
+                        --add-opens java.base/jdk.internal.misc=org.apache.datasketches.memory
+                        --add-exports org.apache.datasketches.memory.tests/org.apache.datasketches.memory.test=org.testng
+                    </argLine>
+                    <jdkToolchain>
+                        <version>9</version>
+                    </jdkToolchain>
+                </configuration>
+            </plugin>
+
+        </plugins>
+    </build>
+
 </project>
diff --git a/datasketches-memory-java9/pom.xml b/datasketches-memory-java9/pom.xml
@@ -43,6 +43,7 @@
         <maven.install.skip>true</maven.install.skip>
         <maven.javadoc.skip>true</maven.javadoc.skip>
         <java.version>9</java.version>
+        <jdk-toolchain.version>9</jdk-toolchain.version>
         <maven.compiler.source>${java.version}</maven.compiler.source>
         <maven.compiler.target>${java.version}</maven.compiler.target>
     </properties>
@@ -55,11 +56,12 @@
                     <artifactId>maven-compiler-plugin</artifactId>
                     <version>${maven-compiler-plugin.version}</version>
                     <configuration>
-                        <source>9</source>
-                        <target>9</target>
-                        <jdkToolchain>
-                            <version>9</version>
-                        </jdkToolchain>
+                        <compilerArgs>
+                            <arg>--add-exports</arg>
+                            <arg>java.base/jdk.internal.ref=org.apache.datasketches.memory</arg>
+                            <arg>--add-exports</arg>
+                            <arg>java.base/jdk.internal.misc=org.apache.datasketches.memory</arg>
+                        </compilerArgs>
                     </configuration>
                 </plugin>
             </plugins>

diff --git a/datasketches-memory-java9/src/main/java/module-info.java b/datasketches-memory-java9/src/main/java/module-info.java
@@ -0,0 +1,27 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+module org.apache.datasketches.memory {
+    requires java.base;
+    requires java.logging;
+    requires jdk.unsupported;
+
+    exports org.apache.datasketches.memory;
+    exports org.apache.datasketches.memory.internal to org.apache.datasketches.memory.tests;
+}
diff --git a/datasketches-memory-java9/src/main/java/org/apache/datasketches/memory/Map.java b/datasketches-memory-java9/src/main/java/org/apache/datasketches/memory/Map.java
@@ -0,0 +1,46 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.datasketches.memory;
+
+/**
+ * Read only interface for a memory mapped file
+ *
+ * @author Roman Leventov
+ * @author Lee Rhodes
+ * @author Praveenkumar Venkatesan
+ */
+public interface Map extends AutoCloseable {
+
+  /**
+   * @see <a href="https://docs.oracle.com/javase/8/docs/api/java/nio/MappedByteBuffer.html#load--">
+   * java/nio/MappedByteBuffer.load</a>
+   */
+  void load();
+
+  /**
+   * @return true if loaded
+   *
+   * @see <a href=
+   * "https://docs.oracle.com/javase/8/docs/api/java/nio/MappedByteBuffer.html#isLoaded--"> java
+   * /nio/MappedByteBuffer.isLoaded</a>
+   */
+  boolean isLoaded();
+
+}
diff --git a/datasketches-memory-java9/src/main/java/org/apache/datasketches/memory/package-info.java b/datasketches-memory-java9/src/main/java/org/apache/datasketches/memory/package-info.java
@@ -0,0 +1,20 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.datasketches.memory;
diff --git a/docs/development.md b/docs/development.md
@@ -0,0 +1,38 @@
+# Development environment setup
+
+The use of Maven submodules to build a Multi Release JAR was motivated by its compatibility with popular IDEs.
+
+There are two configuration properties to be aware of when configuring your local development environment:
+
+#### Java compiler versions
+
+Settings are usually synchronised with maven toolchain configuration, otherwise the Java version for a maven module
+should be set as follows:
+
+| Maven submodule                   | JDK |
+| --------------------------------- | --- |
+| datasketches-memory-root			|  8  |
+| datasketches-memory			    |  8  |
+| datasketches-memory-java8			|  8  |
+| datasketches-memory-java8-tests	|  8  |
+| datasketches-memory-java9		    |  9  |
+| datasketches-memory-java9-tests	|  9  |
+| datasketches-memory-java11		|  11 |
+
+#### Compiler arguments for JPMS
+
+In order to compile Maven modules in Java versions 9 and above, it is necessary to provide the following arguments to the
+compiler.  These are usually synchronised with the `pom.xml` configuration:
+
+```xml
+    <compilerArgs>
+        <arg>--add-exports</arg>
+        <arg>java.base/jdk.internal.ref=org.apache.datasketches.memory</arg>
+        <arg>--add-exports</arg>
+        <arg>java.base/jdk.internal.misc=org.apache.datasketches.memory</arg>
+    </compilerArgs>
+```
+
+### IntelliJ configuration
+
+### Eclipse configuration