Skip to content
Permalink
Browse files
Merge pull request #141 from apache/documentation-updates
Documentation updates
  • Loading branch information
leerho committed Aug 16, 2021
2 parents 2a38445 + 0e0ee40 commit 6d01ebad3aad3f0dc94627938a766b2018b97eb2
Showing 9 changed files with 248 additions and 85 deletions.
@@ -39,56 +39,30 @@ If you are interested in making contributions to this site please see our
---

## Java Support
Datasketches Memory currently supports Java 8 up to and including Java 13.

Datasketches memory currently supports Java 8 up to and including Java 13.

In order to allocate off-heap memory using the library in Java 9 and above, you must provide the
following runtime arguments to the JVM:

```shell
--add-opens java.base/java.nio=org.apache.datasketches.memory \
--add-opens java.base/jdk.internal.misc=org.apache.datasketches.memory \
--add-opens java.base/jdk.internal.ref=org.apache.datasketches.memory
```

For example, to run your local application with full compatibility for the Java module system,
you might use the following command:

```shell
$JAVA \
--module-path mods \
--add-opens java.base/java.nio=org.apache.datasketches.memory \
--add-opens java.base/jdk.internal.misc=org.apache.datasketches.memory \
--add-opens java.base/jdk.internal.ref=org.apache.datasketches.memory \
--module my.main.application.module
```

where `mods` is your module path and `my.main.application.module` is your own JPMS module:

```java
module datasketches.memory.multirelease.test {
requires org.apache.datasketches.memory;
}
```

Note that the `add-opens` arguments are not required for cases where memory is allocated on the
heap.
__NOTE:__ You may have to provide additional JPMS arguments in order to use the library in Java 9 and above,
see the [usage instructions](docs/usage-instructions.md) for details.

---

## Build Instructions
__NOTE:__ This component accesses resource files for testing. As a result, the directory elements
of the full absolute path of the target installation directory must qualify as Java identifiers.
In other words, the directory elements must not have any space characters (or non-Java identifier
characters) in any of the path elements. This is required by the Oracle Java Specification in
order to ensure location-independent access to resources:
[See Oracle Location-Independent Access to Resources](https://docs.oracle.com/javase/8/docs/technotes/guides/lang/resources.html)

__IMPORTANT:__ This project is structured as a maven multi-module project.
Building this project might affect plugins that require early dependency resolution, such as the
javadoc and eclipse plugins. The build instructions below have been modified to use the
`process-classes` phase (instead of `compile`) for these use cases. For more information,
see this [Maven Reactor Issue](https://issues.apache.org/jira/browse/MNG-3283).
__NOTE:__

1) This component accesses resource files for testing. As a result, the directory elements
of the full absolute path of the target installation directory must qualify as Java identifiers.
In other words, the directory elements must not have any space characters (or non-Java identifier
characters) in any of the path elements. This is required by the Oracle Java Specification in
order to ensure location-independent access to resources:
[See Oracle Location-Independent Access to Resources](https://docs.oracle.com/javase/8/docs/technotes/guides/lang/resources.html)

2) This project is structured as a Maven multi-module project.
Building this project might affect plugins that require early dependency resolution, such as the
javadoc and eclipse plugins.
The build instructions below have been modified to use the `process-classes` phase (instead of `compile`)
for these use cases.

For more information, see this [Maven Reactor Issue](https://issues.apache.org/jira/browse/MNG-3283).

### JDK versions required to compile
This DataSketches component is pure Java and requires the following JDKs to compile:
@@ -158,6 +132,7 @@ detail:
* [Maven Toolchains Configuration](docs/maven-toolchains.md)
* [Multi-Release Jar](docs/multi-release-jar.md)
* [Java Platform Module System](docs/module-system.md)
* [Usage instructions](docs/usage-instructions.md)

In order to build and contribute to this project, please read the relevant IDE documentation:

@@ -169,9 +169,10 @@
<configuration>
<skip>false</skip>
<argLine>
--add-exports java.base/jdk.internal.misc=ALL-UNNAMED
--add-exports java.base/jdk.internal.ref=ALL-UNNAMED
--add-opens java.base/java.nio=ALL-UNNAMED
--add-opens java.base/jdk.internal.misc=ALL-UNNAMED
--add-opens java.base/jdk.internal.ref=ALL-UNNAMED
--add-opens java.base/sun.nio.ch=ALL-UNNAMED
</argLine>
<dependenciesToScan>
<dependency>org.apache.datasketches:datasketches-memory-java8-tests</dependency>
@@ -24,6 +24,9 @@ The use of Maven submodules to build a Multi Release JAR was motivated by its co
popular IDEs. There are two configuration properties to be aware of when configuring your local
development environment:

1) Java compiler versions
2) Compiler arguments for JPMS

### Java compiler versions

Settings are usually synchronised with Maven Toolchain configuration, otherwise the Java version
@@ -78,7 +81,7 @@ Please note that this plugin is retired and no longer maintained!

From the **Package Explorer** View:

- Right click on a blank space in the view
- Right-click on a blank space in the view
- Select **Import/Maven/Existing Maven Projects**
- Select **Next**, and browse to the project directory
- Click **Open**
@@ -25,6 +25,9 @@ popular IDEs.
There are two configuration properties to be aware of when configuring your local development
environment:

1) Java compiler versions
2) Compiler arguments for JPMS

#### Java compiler versions

Settings are usually synchronised with maven toolchain configuration, otherwise the Java version
@@ -31,7 +31,7 @@ From the [maven-toolchain-plugin documentation](https://maven.apache.org/plugins
### Motivation

Toolchains are used in different maven modules to ensure that the correct Java compiler version
is used when compiling source files. This is because `datasketches-memory` uses some JDK
is used when compiling source files. This is because Datasketches Memory uses some JDK
version-specific APIs, which require different JDKs to compile correctly.

### Toolchains template
@@ -48,7 +48,7 @@ the requisite entries should be merged into the existing file if they do not alr

### Environment variables

The `dataSketches-memory` component is pure Java and requires the following JDKs to compile:
The DataSketches Memory component is pure Java and requires the following JDKs to compile:

- JDK8/Hotspot
- JDK9/Hotspot
@@ -73,8 +73,7 @@ might be configured as follows:
Users can discover what JDKs have been loaded into their environment by using the following
command:

- /usr/libexec/java_home -V

/usr/libexec/java_home -V

### Eclipse configuration

@@ -83,5 +82,5 @@ appropriate JDK for each module - see the [Eclipse IDE Setup](eclipse.md).

### IntelliJ configuration

Similarly, if you are an Eclipse user, you may need further configuration for your IDE to use the
Similarly, if you are an IntelliJ user, you may need further configuration for your IDE to use the
appropriate JDK for each module - see the [IntelliJ IDE Setup](intellij.md).
@@ -23,7 +23,7 @@ This project is a multi-module Maven project. A multi-module Maven project consi
aggregator project (the `datasketches-memory-root` project), together with a set of submodules.
The aggregator's configuration is inherited by each submodule, thus reducing duplication.

`datasketches-memory` makes use of some features of the Java platform, for example, `Unsafe`,
Datasketches Memory makes use of some features of the Java platform, for example, `Unsafe`,
which have evolved in Java versions 9 and above. Therefore, a multi-module project allows us to
add support for later versions of Java by using independent Maven modules to target
platform-specific APIs. For example, to deallocate references a `sun.misc.Cleaner` will be used
@@ -40,7 +40,7 @@ components, and which are not;
JDK internals are now strongly encapsulated, except for critical internal APIs such as
`sun.misc.Unsafe` (see [JEP-396](https://openjdk.java.net/jeps/396) and
[JEP-403](https://openjdk.java.net/jeps/403)).
`datasketches-memory` can no longer access these APIs by default, and requires explicit access.
Datasketches Memory can no longer access these APIs by default, and requires explicit access.

### Module declarations

@@ -59,36 +59,21 @@ module org.apache.datasketches.memory {
requires jdk.unsupported;
exports org.apache.datasketches.memory;
exports org.apache.datasketches.memory.internal to org.apache.datasketches.memory.tests;
}
```

This declaration explicitly defines the dependencies for `datasketches-memory`, as well as the
This declaration explicitly defines the dependencies for the `org.apache.datasketches.memory` module, as well as the
external API. The `org.apache.datasketches.internal` package is now inaccessible to the end user,
providing better encapsulation.

#### org.apache.datasketches.memory.tests

The module declaration above makes provision for unit testing.
The `org.apache.datasketches.internal` package is not accessible to the end user,
but is accessible to the `org.apache.datasketches.memory.tests` module:

```java
module org.apache.datasketches.memory.tests {
requires java.base;
requires org.testng;
requires org.apache.datasketches.memory;
}
```

### Compiler arguments

Some dependencies are encapsulated by default, and this causes compilation to fail for
Java versions 9 and above.
These dependencies can be made accessible at compile time through the use of the
`add-exports` compiler argument.
This argument allows one module to access some of the unexported types of another module.
Datasketches memory has come to depend on several internal APIs and therefore requires special
Datasketches Memory has come to depend on several internal APIs and therefore requires special
exposition.

For example, in order to compile the `datasketches-memory-java9` submodule, the following compiler
@@ -108,18 +93,11 @@ where fields and methods that do not have `public` visibility in a class.
Reflective access requires additional arguments to be provided by the user at runtime,
in order to use the `datasketches-memory` JPMS module in Java versions 9 and above.

The following runtime arguments should be provided when allocating memory off-heap:

```shell
--add-opens java.base/java.nio=org.apache.datasketches.memory \
--add-opens java.base/jdk.internal.misc=org.apache.datasketches.memory \
--add-opens java.base/jdk.internal.ref=org.apache.datasketches.memory
```

Note that these arguments are not required for cases where memory is allocated on the heap.
See the [usage instructions](usage-instructions.md) for more details.

### JPMS and Java 8

Java 8 does not support module declarations and the JPMS module system.
Java 8 does not support module declarations and the JPMS module system, and no additional
runtime arguments are necessary.
However, support is retained for Java 8 users by only including the compiled declaration
(`module-info.class`) in the `datasketches-memory` multi-release JAR for Java9 and above.
@@ -19,8 +19,8 @@

# Multi-Release JAR

The `datasketches-memory` module assembles a JAR for release that consists of multiple
Java-release-specific versions of class files to coexist in a single archive (the MR-JAR).
The `datasketches-memory` module assembles a multi-release (MR) JAR for release that consists of
multiple Java-release-specific versions of compiled class files.

From [JEP-238](https://openjdk.java.net/jeps/238):

@@ -30,7 +30,8 @@ language or API features available in newer releases since it is difficult to ex
platform dependencies, which generally involves reflection, or to distribute different library
artifacts for different platform versions.
This is specifically the case for the DataSketches Memory component.
The next case describes the challenge in supporting newer versions of Java for libraries
such as DataSketches Memory:

> Some libraries and frameworks, furthermore, use internal APIs of the JDK that will be made
inaccessible in Java 9 when module boundaries are strictly enforced. This also creates a

0 comments on commit 6d01eba

Please sign in to comment.