Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

xgboost extension fails to initialize on JDK 17 due to attempt to use reflection to load native library #16060

Closed
ispringer opened this issue Feb 5, 2024 · 2 comments
Assignees
Labels
Milestone

Comments

@ispringer
Copy link

H2O version, Operating System and Environment

3.44.0.3 w/ xgboost 2.12_1.6.1.24, Mac OS X w/ M1 chip, JDK 17.0.9+9

Actual behavior

WARN  2024-02-05 16:51:57,764 hex.tree.xgboost.util.NativeLibrary  Cannot load library from path lib/osx_64/libxgboost4j_minimal.dylib
WARN  2024-02-05 16:51:57,765 hex.tree.xgboost.util.NativeLibrary  Failed to load library from both native path and jar!
INFO  2024-02-05 16:51:57,765 hex.tree.xgboost.util.NativeLibraryLoaderChain  Cannot load library: xgboost4j_minimal (lib/osx_64/libxgboost4j_minimal.dylib)
ERROR 2024-02-05 16:51:57,766 ai.h2o.xgboost4j.java.NativeLibLoader  Failed to add native path to the classpath at runtime
java.io.IOException: Failed to get field handle to set library path
	at ai.h2o.xgboost4j.java.NativeLibLoader.addNativeDir(NativeLibLoader.java:392)
	at ai.h2o.xgboost4j.java.NativeLibLoader.initXGBoost(NativeLibLoader.java:170)
	at ai.h2o.xgboost4j.java.NativeLibLoader.getLoader(NativeLibLoader.java:193)
	at hex.tree.xgboost.XGBoostExtension.getLoader(XGBoostExtension.java:74)
	at hex.tree.xgboost.XGBoostExtension.initXgboost(XGBoostExtension.java:99)
	at hex.tree.xgboost.XGBoostExtension.isEnabled(XGBoostExtension.java:52)
	at water.ExtensionManager.isEnabled(ExtensionManager.java:192)
	at water.ExtensionManager.registerCoreExtensions(ExtensionManager.java:106)
	at water.H2O.main(H2O.java:2342)
	at water.H2OStarter.start(H2OStarter.java:22)
	at water.H2OStarter.start(H2OStarter.java:52)
	at org.apache.spark.h2o.backends.internal.InternalH2OBackend$.org$apache$spark$h2o$backends$internal$InternalH2OBackend$$startSingleH2OWorker(InternalH2OBackend.scala:146)
	at org.apache.spark.h2o.backends.internal.InternalH2OBackend.startH2OCluster(InternalH2OBackend.scala:45)
	at ai.h2o.sparkling.H2OContext.<init>(H2OContext.scala:82)
	at ai.h2o.sparkling.H2OContext$.getOrCreate(H2OContext.scala:447)

Expected behavior
It should successfully load the xgboost native lib using an approach that is allowed by JDK17 (see https://stackoverflow.com/questions/15409223/adding-new-paths-for-native-libraries-at-runtime-in-java)

Steps to reproduce

  1. Try to initialize H2O with xgboost extension on JDK 17
@ispringer ispringer added the bug label Feb 5, 2024
@valenad1
Copy link
Collaborator

valenad1 commented Feb 9, 2024

Hello @ispringer, XGBoost is not supported on Apple M1 yet.

@valenad1 valenad1 self-assigned this Feb 9, 2024
@ispringer
Copy link
Author

Hi @valenad1,

Thanks, I actually ran my application on Linux with JDK 17 after filing this, and it worked.

The "Failed to get field handle to set library path" made me think it was the JDK 17 issue described in the stackoverflow link. It would be great if the error handling could be improved, so it prints something like "H2O is not supported on operating system OS X with architecture aarch64"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants