Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JRuby can Not load JFFI library under IBM Power 64 with PLinux platform #1389

Closed
xialixl opened this Issue Jan 9, 2014 · 32 comments

Comments

Projects
None yet
8 participants
@xialixl
Copy link

xialixl commented Jan 9, 2014

Environments:
IBM Power ppc 64 bit , POWER7 architecture,
Red Hat Enterprise Linux Server release 6.3 (Santiago)
JRuby ver 1.6.8, jar file: jruby-complete-1.6.8.jar
Apache Hbase 0.95.2

In case of running Command from Hbase :
.../hbase-0.95.2/bin/hbase shell

We got Errors like following:

INFO  [main] Configuration.deprecation: hadoop.native.lib is deprecated. Instead, use io.native.lib.available
java.lang.RuntimeException: java.lang.UnsatisfiedLinkError: Could not locate stub library in jar file.  Tried [jni/ppc64-Linux/libjffi-1.0.so, /jni/ppc64-Linux/libjffi-1.0.so]
        at com.kenai.jffi.Foreign$InValidInstanceHolder.getForeign(Foreign.java:90)

My Analysis:
It looks like JRuby was going to load Java JFFI lib: libjffi-1.0.so which is supposed to exist under location jni/ppc64-Linux/libjffi-1.0.so inside the Jar 'ruby-complete-1.6.8.jar'.
for 1.6.8 version currently Jruby only has following kinds of JFFI lib for choices :

jni/Darwin/
jni/Darwin/libjffi-1.0.jnilib
jni/i386-FreeBSD/
jni/i386-FreeBSD/libjffi-1.0.so
jni/i386-Linux/
jni/i386-Linux/libjffi-1.0.so
jni/i386-OpenBSD/
jni/i386-OpenBSD/libjffi-1.0.so
jni/i386-SunOS/
jni/i386-SunOS/libjffi-1.0.so
jni/i386-Windows/
jni/i386-Windows/jffi-1.0.dll
jni/ppc-AIX/
jni/ppc-AIX/libjffi-1.0.a
jni/ppc-Linux/
jni/ppc-Linux/libjffi-1.0.so
jni/sparc-SunOS/
jni/sparc-SunOS/libjffi-1.0.so
jni/sparcv9-SunOS/
jni/sparcv9-SunOS/libjffi-1.0.so
jni/x86_64-FreeBSD/
jni/x86_64-FreeBSD/libjffi-1.0.so
jni/x86_64-Linux/
jni/x86_64-Linux/libjffi-1.0.so
jni/x86_64-OpenBSD/
jni/x86_64-OpenBSD/libjffi-1.0.so
jni/x86_64-SunOS/
jni/x86_64-SunOS/libjffi-1.0.so
jni/x86_64-Windows/
jni/x86_64-Windows/jffi-1.0.dll

Fix:
Since '/jni/ppc64-Linux/libjffi-1.0.so' is missed inside jar , I downloaded the right version of JFFI sources (https://github.com/jnr/jffi) and built them by IBM XL C Compiler instead of default GCC at IBM ppc64 platform , Added the output libjffi-1.0.so into the right place of the jruby-complete-1.6.8.jar. Now it works well at my ppc64 env.

Advice:
To support IBM ppc64 with Linux env , Could Jruby add JFFI library for ppc64 version into released Jruby jar files.

Thanks

@BanzaiMan

This comment has been minimized.

Copy link
Member

BanzaiMan commented Jan 18, 2014

We generally don't have access to these hardware, so we ask users to provide us with binary files such as these.

Thanks.

@xialixl

This comment has been minimized.

Copy link
Author

xialixl commented Jan 21, 2014

Ok, I got the lib file and jar file that could provide you with jruby, but how I could share with you and the project , Fork the project ?
sorry I am quite new for using github .

@faisalmoeen

This comment has been minimized.

Copy link

faisalmoeen commented Jun 22, 2014

I am facing the same issue. Xialixl, can you please send the files to fmorakzai@gmail.com ?

@mkristian

This comment has been minimized.

Copy link
Member

mkristian commented Jun 22, 2014

@xialixl sorry for the late reply: fork https://github.com/jnr/jffi and send a pull request

@ayappanec

This comment has been minimized.

Copy link

ayappanec commented Sep 9, 2014

A pull request has been sent to jffi repo with both ppc64-Linux and ppc64le-Linux binaries. Now the question here is whether the binaries 'ppc64 (ppc64le) libjffi-1.0.so' files will be added to the already published jruby-complete-1.6.8.jar file ?

@mkristian

This comment has been minimized.

Copy link
Member

mkristian commented Sep 9, 2014

we can not republish released jar (at least on maven central this is not
possible), but if you have a local maven repo you could add the missing
libjffi.so in the jruby-complete file

@ayappanec

This comment has been minimized.

Copy link

ayappanec commented Sep 9, 2014

Ok. We have this problem of locally building jruby-complete-1.6.8.jar file in ppc64 (Big Endian) systems
A issue has been opened ( https://jira.codehaus.org/browse/JRUBY-7201 )

@ayappanec

This comment has been minimized.

Copy link

ayappanec commented Jan 23, 2015

Can any maintainers look into this issue ( jnr/jffi#12 )

@headius

This comment has been minimized.

Copy link
Member

headius commented Jan 23, 2015

Wow, apologies for missing this one for so long. The PR has been merged and we'll try to get releases of the whole JNR chain out for JRuby 1.7.x and 9k.pre2.

@headius headius closed this Jan 23, 2015

@headius headius added this to the 9.0.0.0.pre2 milestone Jan 23, 2015

@headius

This comment has been minimized.

Copy link
Member

headius commented Jan 23, 2015

Oops...don't close until we have updated JNR-* in a JRuby release.

@headius headius reopened this Jan 23, 2015

@ayappanec

This comment has been minimized.

Copy link

ayappanec commented Jan 28, 2015

Can this be added to upcoming JRuby 1.7.x release also ?

@ayappanec

This comment has been minimized.

Copy link

ayappanec commented Feb 4, 2015

Any update for the above comment?

@gustavotemple

This comment has been minimized.

Copy link
Contributor

gustavotemple commented Feb 25, 2015

@ayappanec, did you see this issue?

jnr/jffi#15

@headius

This comment has been minimized.

Copy link
Member

headius commented Feb 25, 2015

I don't see why we couldn't update this in 1.7.x as well. @enebo and I know we need to update everything for next releases, which will involve fixing some regressions in the jnr-* chain of projects. Marking for 1.7.20.

@headius headius modified the milestones: JRuby 1.7.20, 9.0.0.0.pre2 Feb 25, 2015

@headius

This comment has been minimized.

Copy link
Member

headius commented Feb 26, 2015

jnr/jffi#16 details the now-mitigated showstopper preventing us from releaseing jffi 1.2.8. Should be able to get the family released this week.

@ayappanec

This comment has been minimized.

Copy link

ayappanec commented Mar 2, 2015

I came to know about the problem you guys facing for releasing jffi 1.2.8. I hope this one will get into Jruby 1.7.20 & 9000. Thank you.

@enebo

This comment has been minimized.

Copy link
Member

enebo commented Mar 3, 2015

@headius was able to release jffi and dependent artifacts yesterday so the plan is to update master and jruby-1_7 before our next releases.

@headius

This comment has been minimized.

Copy link
Member

headius commented Mar 3, 2015

1.7 and master have both been updated for latest jnr-* projects.

If it's possible for someone on PLinux/ppc64 to do a build of jruby-1_7 branch and confirm it works properly, we'd appreciate it :-)

@headius headius closed this Mar 3, 2015

@ayappanec

This comment has been minimized.

Copy link

ayappanec commented Mar 4, 2015

Building jruby-1_7 branch fails in PLinux/ppc64 & PLinux/ppc64le due to unknown compilation issue.
"Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project jruby-core: Compilation failure"

But the main problem here is we are not able to build jruby-complete jar individually due to missing artfifacts.
"Failed to execute goal on project jruby-complete: Could not resolve dependencies for project org.jruby:jruby-complete:bundle:1.7.20-SNAPSHOT: The following artifacts could not be resolved: org.jruby:jruby-core:jar:noasm:1.7.20-SNAPSHOT, org.jruby:jruby-stdlib:jar:1.7.20-SNAPSHOT: Could not find artifact org.jruby:jruby-core:jar:noasm:1.7.20-SNAPSHOT in sonatype (https://oss.sonatype.org/content/repositories/snapshots/)"

It seems like the artifacts are not yet available in sonatype repository.

@mkristian

This comment has been minimized.

Copy link
Member

mkristian commented Mar 4, 2015

to build jruby-complete you need to run from the basedir of jruby
mvn -Pcomplete
or mvn clean install -Pcomplete
if this is not passing beyond jruby-core then nothing furtehr down the road
will work since all those missing artifacts will be build on the way

On Wed, Mar 4, 2015 at 8:25 AM, ayappanec notifications@github.com wrote:

Building jruby-1_7 branch fails in PLinux/ppc64 & PLinux/ppc64le due to
unknown compilation issue.
"Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.1:compile
(default-compile) on project jruby-core: Compilation failure"

But the main problem here is we are not able to build jruby-complete jar
individually due to missing artfifacts.
"Failed to execute goal on project jruby-complete: Could not resolve
dependencies for project org.jruby:jruby-complete:bundle:1.7.20-SNAPSHOT:
The following artifacts could not be resolved:
org.jruby:jruby-core:jar:noasm:1.7.20-SNAPSHOT,
org.jruby:jruby-stdlib:jar:1.7.20-SNAPSHOT: Could not find artifact
org.jruby:jruby-core:jar:noasm:1.7.20-SNAPSHOT in sonatype (
https://oss.sonatype.org/content/repositories/snapshots/)"

It seems like the artifacts are not yet available in sonatype repository.


Reply to this email directly or view it on GitHub
#1389 (comment).

@ayappanec

This comment has been minimized.

Copy link

ayappanec commented Mar 4, 2015

I tried both the options but they failed to get beyond jruby-core. Finally i went into the maven/jruby-complete directory and tried to build jruby-complete jar by issuing "mvn clean install" and there i faced this missing artifacts issue..

@mkristian

This comment has been minimized.

@headius

This comment has been minimized.

Copy link
Member

headius commented Mar 4, 2015

@ayappanec Please open a separate issue for your problems building JRuby on PLinux and we'll work through them separately. I suspect it's some oddity in how our build interacts with the JVM on that platform.

@ayappanec

This comment has been minimized.

Copy link

ayappanec commented Mar 4, 2015

@mkristian It requires jruby-core,jruby-stdlib and other such jars also.Can you please place those jars also in sonatype?
@headius Sure. I will open a separate issue for building JRuby on PLinux

@mkristian

This comment has been minimized.

Copy link
Member

mkristian commented Mar 4, 2015

@ayappanec

This comment has been minimized.

Copy link

ayappanec commented Mar 4, 2015

I tried with the command mentioned above. its trying to download the jruby-core jar from sonatype but failed since the jar uploaded there has some number associated with the filename "jruby-stdlib-1.7.20-20150304.132127-1.jar"

@headius

This comment has been minimized.

Copy link
Member

headius commented Mar 4, 2015

@ayappanec Thank you...please let us know when you've opened the other issue. For confirming this issue right now, just download the 1.7.20 build that @mkristian provided above.

@ayappanec

This comment has been minimized.

Copy link

ayappanec commented Mar 5, 2015

@headius I could see the jffi binaries for ppc64 & ppc64le in the jruby-complete jar. Thank you

@ayappanec

This comment has been minimized.

Copy link

ayappanec commented Mar 5, 2015

@headius I opened a new issue for Jruby build failure (#2648)

@ayappanec

This comment has been minimized.

Copy link

ayappanec commented Mar 31, 2015

It seems like we need ppc64le jnr-ffi TypeAliases and ruby ffi ppc64le platform specific types.conf as well to get the latest jruby working ppc64le. I could see the changes already went into the master branch of jnr-ffi & jruby. I don't know when the jnr-ffi next release is going to happen. If it happens before jruby release, then we need to change the jnr-ffi version to 2.0.3 in jruby to make use of it. Otherwise we have wait till further releases of jruby.

@headius

This comment has been minimized.

Copy link
Member

headius commented Apr 6, 2015

@ayappanec I'm largely paralyzed from improving ppc64le support right now without your help, so if there are additional issues we need to fix, I'm hoping to see PRs from you :-)

@ayappanec

This comment has been minimized.

Copy link

ayappanec commented Apr 7, 2015

@headius Sure.. Right now #2790 & #1897 seems to be the major hurdle. #2647 is yet another one but we have openjdk as an option here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.