Skip to content

Commit

Permalink
ARROW-16566: [Java] Initialize JNI components on use instead of stati…
Browse files Browse the repository at this point in the history
…cally (apache#13146)

We are using Arrow in Spark. When speculation is turned on, it is possible for the task to be forcibly killed during the running process. It means, `JniLoader.get().ensureLoaded()` maybe failed with `ClosedByInterruptException` when the task was killed, and it should be accepted with this.
However, `NativeMemoryPool` use it as a static code, this leads to an init failure every time the NativeMemroyPool is trying to initialized and can never succeed. It would always fail to init `NativeMemoryPool` with `java.lang.NoClassDefFoundError: Could not initialize class org.apache.arrow.dataset.jni.NativeMemoryPool`.

Lead-authored-by: jackylee-ch <lijunqing@baidu.com>
Co-authored-by: stczwd <qcsd2011@163.com>
Signed-off-by: David Li <li.davidm96@gmail.com>
  • Loading branch information
jackylee-ch and jackylee-ch committed Jun 23, 2022
1 parent 71cad74 commit 4c61f99
Show file tree
Hide file tree
Showing 3 changed files with 4 additions and 8 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -28,11 +28,11 @@ public class JniWrapper {
private static final JniWrapper INSTANCE = new JniWrapper();

public static JniWrapper get() {
JniLoader.get().ensureLoaded();
return INSTANCE;
}

private JniWrapper() {
JniLoader.get().ensureLoaded();
}

/**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,11 +25,11 @@ public class JniWrapper {
private static final JniWrapper INSTANCE = new JniWrapper();

public static JniWrapper get() {
JniLoader.get().ensureLoaded();
return INSTANCE;
}

private JniWrapper() {
JniLoader.get().ensureLoaded();
}

/**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,6 @@
public class NativeMemoryPool implements AutoCloseable {
private final long nativeInstanceId;

private static void ensureJniLoaded() {
JniLoader.get().ensureLoaded();
}

private NativeMemoryPool(long nativeInstanceId) {
this.nativeInstanceId = nativeInstanceId;
}
Expand All @@ -35,7 +31,7 @@ private NativeMemoryPool(long nativeInstanceId) {
* Get the default memory pool. This will return arrow::default_memory_pool() directly.
*/
public static NativeMemoryPool getDefault() {
ensureJniLoaded();
JniLoader.get().ensureLoaded();
return new NativeMemoryPool(getDefaultMemoryPool());
}

Expand All @@ -45,7 +41,7 @@ public static NativeMemoryPool getDefault() {
* from the listener in advance.
*/
public static NativeMemoryPool createListenable(ReservationListener listener) {
ensureJniLoaded();
JniLoader.get().ensureLoaded();
return new NativeMemoryPool(createListenableMemoryPool(listener));
}

Expand Down

0 comments on commit 4c61f99

Please sign in to comment.