Run BookKeeper on Java9

[ivankelly]

JIRA: https://issues.apache.org/jira/browse/BOOKKEEPER-1049

Reporter: Enrico Olivelli @eolivelli

I am tring to run BookKeeper on Java9, a part from simple issues:

• exclude the transitive dependency to jdk.tools in bookkeeer-benchmarks
• fix a warning due to the fact the Class.forName in java9 is deprecated and we are running javac with the "Werror" option (warning = error)

we have a real stopper, in fact we want to access the low level ID of FileDescriptors using the reflaction and by default this is not allowed.

This is the error:

{code}
java.lang.AssertionError
at org.apache.bookkeeper.util.NativeIO.getFieldByReflection(NativeIO.java:63)
at org.apache.bookkeeper.util.NativeIO.getSysFileDescriptor(NativeIO.java:75)
at org.apache.bookkeeper.bookie.JournalChannel.(JournalChannel.java:218)
at org.apache.bookkeeper.bookie.JournalChannel.(JournalChannel.java:97)
at org.apache.bookkeeper.bookie.JournalChannel.(JournalChannel.java:86)
at org.apache.bookkeeper.bookie.UpgradeTest.writeJournal(UpgradeTest.java:83)
at org.apache.bookkeeper.bookie.UpgradeTest.newV1JournalDirectory(UpgradeTest.java:114)
at org.apache.bookkeeper.bookie.UpgradeTest.testUpgradeV1toCurrent(UpgradeTest.java:192)
{code}

Comments from JIRA

---

Enrico Olivelli 2017-05-08T13:11:05.656+0000

I am investigating if there is another way to access the information

---

Enrico Olivelli 2017-05-09T06:45:14.292+0000

The best solution is to use Native Code to access the internal "fd" field inside the FieldDescriptor object:

• we need the 'fd' only if we can load our native library
• accessing object internals using JNI is allowed by JigSaw

http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-May/047510.html

I will create a patch based on the work of BOOKKEEPER-816 which introduces a BookKeeper specific native-io library

[ivankelly]

@eolivelli @sijie This can be closed, right?

[eolivelli]

@ivankelly @sijie I am going to run (embedded) bookies and clients on Java 9 in production within the end of current month with version 4.6.1.

I would like to close this issue once my team is happy.

[eolivelli]

The issue about FileDescriptor#fd is still here but the JVM only logs a warning (Illegal reflective access...).
We should fix it in the mid term but it need native code

[sijie]

The issue about FileDescriptor#fd is still here but the JVM only logs a warning (Illegal reflective access...).

I am not sure if we can really fix it. because we can't drop it for java 8 users, so this code would remain as there until we don't support java 8.

[eolivelli]

The best way would be to check if we are on jdk9 or above and in that case do not use reflection but use JNI. This is the way JDK team suggested to me.
Anyway bookie runs fine on jdk 9, jdk10 and hopefully even on 11. It will work till JDK guys decide to prevent illegal access to internals of the JDK or change FileDescriptor class.
We have this problem only for a single feature, disabled by default in our code base.
Netty has the same issue but they will fix when they will need it.
I think we can close this issue and resume the thread next year.