Skip to content
Permalink
Browse files
Add license headers.
Compacted line lengths to make raw text more readable. Other minor
edits.
  • Loading branch information
leerho committed Aug 7, 2021
1 parent 6b92a68 commit bdbb636dce71ff4b8033ac9dfbc60c3c20f59fa7
Show file tree
Hide file tree
Showing 7 changed files with 314 additions and 131 deletions.
@@ -26,31 +26,33 @@
=================

# DataSketches Java Memory Component
The goal of this component of the library is to provide high performance access to native memory for primitives
and primitive arrays. It also provides consistent views into heap-based primitive arrays,
Java ByteBuffers and memory mapped files. This package is general purpose, has minimal external
runtime dependencies and can be used in any application that needs to manage data structures outside
the Java heap.
The goal of this component of the library is to provide high performance access to heap memory or
native memory for primitives, primitive arrays, ByteBuffers and memory mapped files.
This package is general purpose, has minimal external runtime dependencies and can be used in any
application that needs to manage data structures outside the Java heap.

Please visit the main [DataSketches website](https://datasketches.apache.org) for more information.

If you are interested in making contributions to this site please see our [Community](https://datasketches.apache.org/docs/Community/) page for how to contact us.
If you are interested in making contributions to this site please see our
[Community](https://datasketches.apache.org/docs/Community/) page for how to contact us.

---

## Java Support

Datasketches memory currently supports Java 8 up to and including Java 13.

In order to allocate off-heap memory using the library in Java 9 and above, you must provide the following runtime arguments to the JVM:
In order to allocate off-heap memory using the library in Java 9 and above, you must provide the
following runtime arguments to the JVM:

```shell
--add-opens java.base/java.nio=org.apache.datasketches.memory \
--add-opens java.base/jdk.internal.misc=org.apache.datasketches.memory \
--add-opens java.base/jdk.internal.ref=org.apache.datasketches.memory
```

For example, to run your local application with full compatibility for the Java module system, you might use the following command:
For example, to run your local application with full compatibility for the Java module system,
you might use the following command:

```shell
$JAVA \
@@ -69,25 +71,38 @@ module datasketches.memory.multirelease.test {
}
```

Note that the `add-opens` arguments are not required for cases where memory is allocated on the heap.
Note that the `add-opens` arguments are not required for cases where memory is allocated on the
heap.

---

## Build Instructions
__NOTE:__ This component accesses resource files for testing. As a result, the directory elements of the full absolute path of the target installation directory must qualify as Java identifiers. In other words, the directory elements must not have any space characters (or non-Java identifier characters) in any of the path elements. This is required by the Oracle Java Specification in order to ensure location-independent access to resources: [See Oracle Location-Independent Access to Resources](https://docs.oracle.com/javase/8/docs/technotes/guides/lang/resources.html)

__IMPORTANT:__ This project is structured as a maven multi-module project. Building this project might affect plugins that require early dependency resolution, such as the javadoc and eclipse plugins. The build instructions below have been modified to use the `process-classes` phase (instead of `compile`) for these use cases. For more information, see [this Maven Reactor issue](https://issues.apache.org/jira/browse/MNG-3283).
__NOTE:__ This component accesses resource files for testing. As a result, the directory elements
of the full absolute path of the target installation directory must qualify as Java identifiers.
In other words, the directory elements must not have any space characters (or non-Java identifier
characters) in any of the path elements. This is required by the Oracle Java Specification in
order to ensure location-independent access to resources:
[See Oracle Location-Independent Access to Resources](https://docs.oracle.com/javase/8/docs/technotes/guides/lang/resources.html)

__IMPORTANT:__ This project is structured as a maven multi-module project.
Building this project might affect plugins that require early dependency resolution, such as the
javadoc and eclipse plugins. The build instructions below have been modified to use the
`process-classes` phase (instead of `compile`) for these use cases. For more information,
see this [Maven Reactor Issue](https://issues.apache.org/jira/browse/MNG-3283).

### JDK versions required to compile
This DataSketches component is pure Java and requires the following JDKs to compile:

- JDK8/Hotspot
- JDK9/Hotspot
- JDK11/Hotspot

Ensure that your local environment has been configured according to the [Maven toolchain configuration](docs/maven-toolchains.md).
Ensure that your local environment has been configured according to the
[Maven Toolchains Configuration](docs/maven-toolchains.md).

### Recommended Build Tool
This DataSketches component is structured as a Maven project and Maven is the recommended Build Tool.
This DataSketches component is structured as a Maven project and Maven is the recommended Build
Tool.

There are two types of tests: normal unit tests and tests run by the strict profile.

@@ -121,8 +136,9 @@ This will create the following Jars:

### Toolchains

This project makes use of Maven toolchains to ensure that the correct Java compiler version is used when compiling source files.
See the [Maven toolchain configuration](docs/maven-toolchains.md) for more details.
This project makes use of Maven toolchains to ensure that the correct Java compiler version is
used when compiling source files.
See the [Maven Toolchains Configuration](docs/maven-toolchains.md) for more details.

### Dependencies

@@ -135,14 +151,15 @@ See the pom.xml file for test dependencies.

## Further documentation for contributors

For more information on the project configuration, the following topics are discussed in more detail:
For more information on the project configuration, the following topics are discussed in more
detail:

* [Maven configuration](docs/maven.md)
* [Maven toolchain configuration](docs/maven-toolchains.md)
* [Multi-release jar](docs/multi-release-jar.md)
* [Maven Configuration](docs/maven.md)
* [Maven Toolchains Configuration](docs/maven-toolchains.md)
* [Multi-Release Jar](docs/multi-release-jar.md)
* [Java Platform Module System](docs/module-system.md)

In order to build and contribute to this project, please read the relevant IDE documentation:

- [Eclipse IDE](docs/eclipse.md)
- [IntelliJ IDE](docs/intellij.md)
- [Eclipse IDE Setup](docs/eclipse.md)
- [IntelliJ IDE Setup](docs/intellij.md)
@@ -1,27 +1,49 @@
# Eclipse IDE setup
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
The use of Maven submodules to build a Multi Release JAR was motivated by its compatibility with popular IDEs. There are two configuration properties to be aware of when
configuring your local development environment:
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->


# Eclipse IDE Setup

The use of Maven submodules to build a Multi Release JAR was motivated by its compatibility with
popular IDEs. There are two configuration properties to be aware of when configuring your local
development environment:

### Java compiler versions

Settings are usually synchronised with Maven Toolchain configuration, otherwise the Java version for a Maven module
should be set as follows:
Settings are usually synchronised with Maven Toolchain configuration, otherwise the Java version
for a Maven module should be set as follows:

| Maven submodule | JDK |
| --------------------------------- | --- |
| datasketches-memory-root | 8 |
| datasketches-memory | 8 |
| datasketches-memory-java8 | 8 |
| datasketches-memory-java8-tests | 8 |
| datasketches-memory-java9 | 9 |
| datasketches-memory-java11 | 11 |
| datasketches-memory-root | 8 |
| datasketches-memory | 8 |
| datasketches-memory-java8 | 8 |
| datasketches-memory-java8-tests | 8 |
| datasketches-memory-java9 | 9 |
| datasketches-memory-java11 | 11 |
| datasketches-memory-resources | 8 |

### Compiler arguments for JPMS

In order to compile Maven modules in Java versions 9 and above, it is necessary to provide the following arguments to the
compiler. These are usually synchronised with the `pom.xml` configuration:
In order to compile Maven modules in Java versions 9 and above, it is necessary to provide the
following arguments to the compiler. These are usually synchronised with the `pom.xml`
configuration:

```xml
<compilerArgs>
@@ -38,11 +60,15 @@ Note that the following configuration was verified using Eclipse Version: 2020-1

### The eclipse maven plugin

The [Eclipse Maven plugin](https://maven.apache.org/plugins/maven-eclipse-plugin/) is used to generate Eclipse IDE files. In order to run the eclipse plugin use:
The [Eclipse Maven plugin](https://maven.apache.org/plugins/maven-eclipse-plugin/) is used to
generate Eclipse IDE files. In order to run the eclipse plugin use:

$ mvn clean process-classes eclipse:eclipse -DskipTests=true

More information about using the eclipse plugin with multi-module Maven builds can be found [here](https://maven.apache.org/plugins/maven-eclipse-plugin/reactor.html).
More information about using the eclipse plugin with multi-module Maven builds can be found
in the Maven
[Multiple Module Projects](https://maven.apache.org/plugins/maven-eclipse-plugin/reactor.html)
document.

Please note that this plugin is retired and no longer maintained!

@@ -61,13 +87,17 @@ From the **Package Explorer** View:

### Setting compiler arguments for JPMS

Although these should be set automatically, the Eclipse IDE does not currently configure these settings according to the `pom.xml` - see this [Eclipse Bug](https://github.com/eclipse-m2e/m2e-core/issues/129).
Although these should be set automatically, the Eclipse IDE does not currently configure these
settings according to the `pom.xml` - see this
[Eclipse Bug](https://github.com/eclipse-m2e/m2e-core/issues/129).
Ensure that the required JPMS arguments are set for the compiler (Java 9 only).

- First, right-click on the `datasketches-memory-java9` project, and select **Properties/Java Build Path**.
- First, right-click on the `datasketches-memory-java9` project, and select
**Properties/Java Build Path**.
- Next, open the **Module Dependencies** tab and select the `java.base` package.
- Click on **Configured details**, followed by **Expose package**.
- In the dialog box, enter package: ```jdk.internal.ref```, and `org.apache.datasketches.memory` as the target module.
- In the dialog box, enter package: ```jdk.internal.ref```, and
`org.apache.datasketches.memory` as the target module.
- Ensure that the **exports** checkbox is selected.

![Eclipse java compiler arguments](img/eclipse-java-compiler-arguments-1.png "Eclipse project compiler arguments")
@@ -82,7 +112,8 @@ Note: These arguments need only be supplied for `datasketches-memory-java9`.

### Setting Java compiler settings

This should be set automatically by the IDE. However, you may ensure that the correct Java compliance level is set for each module by using the Eclipse `Java Compiler` dialog.
This should be set automatically by the IDE. However, you may ensure that the correct Java
compliance level is set for each module by using the Eclipse `Java Compiler` dialog.

- Open the **Java Compiler** dialog, and ensure **Enable project specific settings** is checked:

@@ -98,24 +129,29 @@ You might need to verify this for each module, making sure the correct complianc

### Setting JRE library versions

This should be set automatically by the IDE. However, you may ensure that the correct JRE is used for each module by using the Eclipse **Java Build Path** dialog.
This should be set automatically by the IDE. However, you may ensure that the correct JRE is
used for each module by using the Eclipse **Java Build Path** dialog.

- First, open the project properties dialog for the `datasketches-memory-java9` project, and click on **Java Build Path**.
- First, open the project properties dialog for the `datasketches-memory-java9` project, and
click on **Java Build Path**.
- Next, open the **Libraries** tab and select the **JRE System Library** under **Modulepath**.
- Click **Edit** and ensure that the **Execution Environment** is selected and set to Java 9:

![Eclipse build path](img/eclipse-build-path-1.png "Java 9 Eclipse project build path")

- Follow a similar process for `datasketches-memory-java11`, and verify that **Execution Environment** is selected and set to Java 11:
- Follow a similar process for `datasketches-memory-java11`, and verify that
**Execution Environment** is selected and set to Java 11:

![Eclipse build path](img/eclipse-build-path-2.png "Java 11 Eclipse project build path")

- Lastly, for all other modules, verify that the **Execution Environment** is selected and set to the Java 8 JRE:
- Lastly, for all other modules, verify that the **Execution Environment** is selected and set
to the Java 8 JRE:

![Eclipse build path](img/eclipse-build-path-3.png "Java 8 Eclipse project build path")

### Running unit tests

- Under the `datasketches-memory-java-8-tests` module, right-click on the `src/test/java` directory.
- Under the `datasketches-memory-java-8-tests` module, right-click on the `src/test/java`
directory.
- Select **Run-As** / **TestNG Test**
- It should open a new window and run over 400 tests without error.
@@ -1,28 +1,50 @@
# IntelliJ IDE setup
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
The use of Maven submodules to build a Multi Release JAR was motivated by its compatibility with popular IDEs.
http://www.apache.org/licenses/LICENSE-2.0
There are two configuration properties to be aware of when configuring your local development environment:
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

# IntelliJ IDE Setup

The use of Maven submodules to build a Multi Release JAR was motivated by its compatibility with
popular IDEs.

There are two configuration properties to be aware of when configuring your local development
environment:

#### Java compiler versions

Settings are usually synchronised with maven toolchain configuration, otherwise the Java version for a maven module
should be set as follows:
Settings are usually synchronised with maven toolchain configuration, otherwise the Java version
for a maven module should be set as follows:

| Maven submodule | JDK |
| --------------------------------- | --- |
| datasketches-memory-root | 8 |
| datasketches-memory | 8 |
| datasketches-memory-java8 | 8 |
| datasketches-memory-java8-tests | 8 |
| datasketches-memory-java9 | 9 |
| datasketches-memory-java11 | 11 |
| datasketches-memory-root | 8 |
| datasketches-memory | 8 |
| datasketches-memory-java8 | 8 |
| datasketches-memory-java8-tests | 8 |
| datasketches-memory-java9 | 9 |
| datasketches-memory-java11 | 11 |
| datasketches-memory-resources | 8 |

#### Compiler arguments for JPMS

In order to compile Maven modules in Java versions 9 and above, it is necessary to provide the following arguments to the
compiler. These are usually synchronised with the `pom.xml` configuration:
In order to compile Maven modules in Java versions 9 and above, it is necessary to provide the
following arguments to the compiler. These are usually synchronised with the `pom.xml`
configuration:

```xml
<compilerArgs>
@@ -35,7 +57,8 @@ compiler. These are usually synchronised with the `pom.xml` configuration:

## Running Datasketches-Memory in IntelljJ-IDEA

Note that the following configuration was verified using IntelliJ IDEA 2021.1.2 (Community Edition).
Note that the following configuration was verified using IntelliJ IDEA 2021.1.2
(Community Edition).

### Java compiler versions

@@ -47,6 +70,7 @@ Ensure that the correct SDK is used for each module using the IntelliJ project s

### Compiler arguments for JPMS

Ensure that the required JPMS arguments are set for the compiler (Java 9 only). These should be detected and set automatically based on the `pom.xml` configuration.
Ensure that the required JPMS arguments are set for the compiler (Java 9 only).
These should be detected and set automatically based on the `pom.xml` configuration.

![IntelliJ java compiler arguments](img/intellij-java-compiler-arguments.png "Intellij project compiler arguments")

0 comments on commit bdbb636

Please sign in to comment.