-
Notifications
You must be signed in to change notification settings - Fork 5.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
No support for ARM based architectures (Missing hadoop native lib) #17200
Comments
Hi @v-jizhang here is another user request on the same topic we discussed last week. Do you think you can take it as a priority? Thanks! |
Discussed this with @imjalpreet,
|
|
Why is Hadoop 3 a requirement for native ARM libraries? Is it not feasible to recompile the native libs on Hadoop 2.x? Additionally, if we were to move to Hadoop 3, why not consider Hadoop 3.3.x since it upgrades the protobuf dependency from 2.5.0 to 3.7.1? This adds significant value since protobuf 2.5.0 is unsupported/deprecated and doesn't even build Hadoop libs out of box (requires headers for atomics). Hadoop 3.3.0 Release Notes: https://hadoop.apache.org/docs/r3.3.0/hadoop-project-dist/hadoop-common/release/3.3.0/RELEASENOTES.3.3.0.html |
@v-jizhang - It would be great if we can take this on priority. We would like to move our Presto On Spark clusters to AWS Graviton and are waiting for presto to support ARM64. |
i m still facing issue chip : Apple M1 Pro but issue : Screenshot 2024-05-04 at 10 01 10 AM Downloaded the library from the provided link and saved it to a suitable directory, say /Users/ak/lib. Set the DYLD_LIBRARY_PATH environment variable to include the directory where saved the libhadoop.dylib file: export DYLD_LIBRARY_PATH=/Users/ak/lib |
Currently, there prestodb doesn't support ARM architectures for two reasons. The first is the PrestoSystemRequirements class which doesn't include "aarch64" in the allowed os.arch values. The second is that the underlying dependency for our hadoop jars is from 'com.facebook.presto.hadoop' which doesn't currently include nativelibs/ for the aarch64 ARM types. This leaves two options:
Option 1: Have Facebook update their hadoop libraries to also build nativelibs/ for "aarch64"
Option 2: Change the dependency on hadoop packaging to be some other package that does bundle nativelibs/ for "aarch64"
The text was updated successfully, but these errors were encountered: