Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JRuby can Not load JFFI library under IBM Power 64 with PLinux platform #1389

Closed
xialixl opened this issue Jan 9, 2014 · 32 comments
Closed

JRuby can Not load JFFI library under IBM Power 64 with PLinux platform #1389

xialixl opened this issue Jan 9, 2014 · 32 comments

Comments

@xialixl
Copy link

@xialixl xialixl commented Jan 9, 2014

Environments:
IBM Power ppc 64 bit , POWER7 architecture,
Red Hat Enterprise Linux Server release 6.3 (Santiago)
JRuby ver 1.6.8, jar file: jruby-complete-1.6.8.jar
Apache Hbase 0.95.2

In case of running Command from Hbase :
.../hbase-0.95.2/bin/hbase shell

We got Errors like following:

INFO  [main] Configuration.deprecation: hadoop.native.lib is deprecated. Instead, use io.native.lib.available
java.lang.RuntimeException: java.lang.UnsatisfiedLinkError: Could not locate stub library in jar file.  Tried [jni/ppc64-Linux/libjffi-1.0.so, /jni/ppc64-Linux/libjffi-1.0.so]
        at com.kenai.jffi.Foreign$InValidInstanceHolder.getForeign(Foreign.java:90)

My Analysis:
It looks like JRuby was going to load Java JFFI lib: libjffi-1.0.so which is supposed to exist under location jni/ppc64-Linux/libjffi-1.0.so inside the Jar 'ruby-complete-1.6.8.jar'.
for 1.6.8 version currently Jruby only has following kinds of JFFI lib for choices :

jni/Darwin/
jni/Darwin/libjffi-1.0.jnilib
jni/i386-FreeBSD/
jni/i386-FreeBSD/libjffi-1.0.so
jni/i386-Linux/
jni/i386-Linux/libjffi-1.0.so
jni/i386-OpenBSD/
jni/i386-OpenBSD/libjffi-1.0.so
jni/i386-SunOS/
jni/i386-SunOS/libjffi-1.0.so
jni/i386-Windows/
jni/i386-Windows/jffi-1.0.dll
jni/ppc-AIX/
jni/ppc-AIX/libjffi-1.0.a
jni/ppc-Linux/
jni/ppc-Linux/libjffi-1.0.so
jni/sparc-SunOS/
jni/sparc-SunOS/libjffi-1.0.so
jni/sparcv9-SunOS/
jni/sparcv9-SunOS/libjffi-1.0.so
jni/x86_64-FreeBSD/
jni/x86_64-FreeBSD/libjffi-1.0.so
jni/x86_64-Linux/
jni/x86_64-Linux/libjffi-1.0.so
jni/x86_64-OpenBSD/
jni/x86_64-OpenBSD/libjffi-1.0.so
jni/x86_64-SunOS/
jni/x86_64-SunOS/libjffi-1.0.so
jni/x86_64-Windows/
jni/x86_64-Windows/jffi-1.0.dll

Fix:
Since '/jni/ppc64-Linux/libjffi-1.0.so' is missed inside jar , I downloaded the right version of JFFI sources (https://github.com/jnr/jffi) and built them by IBM XL C Compiler instead of default GCC at IBM ppc64 platform , Added the output libjffi-1.0.so into the right place of the jruby-complete-1.6.8.jar. Now it works well at my ppc64 env.

Advice:
To support IBM ppc64 with Linux env , Could Jruby add JFFI library for ppc64 version into released Jruby jar files.

Thanks

@BanzaiMan
Copy link
Member

@BanzaiMan BanzaiMan commented Jan 18, 2014

We generally don't have access to these hardware, so we ask users to provide us with binary files such as these.

Thanks.

@xialixl
Copy link
Author

@xialixl xialixl commented Jan 21, 2014

Ok, I got the lib file and jar file that could provide you with jruby, but how I could share with you and the project , Fork the project ?
sorry I am quite new for using github .

@faisalmoeen
Copy link

@faisalmoeen faisalmoeen commented Jun 22, 2014

I am facing the same issue. Xialixl, can you please send the files to fmorakzai@gmail.com ?

@mkristian
Copy link
Member

@mkristian mkristian commented Jun 22, 2014

@xialixl sorry for the late reply: fork https://github.com/jnr/jffi and send a pull request

@ayappanec
Copy link

@ayappanec ayappanec commented Sep 9, 2014

A pull request has been sent to jffi repo with both ppc64-Linux and ppc64le-Linux binaries. Now the question here is whether the binaries 'ppc64 (ppc64le) libjffi-1.0.so' files will be added to the already published jruby-complete-1.6.8.jar file ?

@mkristian
Copy link
Member

@mkristian mkristian commented Sep 9, 2014

we can not republish released jar (at least on maven central this is not
possible), but if you have a local maven repo you could add the missing
libjffi.so in the jruby-complete file

@ayappanec
Copy link

@ayappanec ayappanec commented Sep 9, 2014

Ok. We have this problem of locally building jruby-complete-1.6.8.jar file in ppc64 (Big Endian) systems
A issue has been opened ( https://jira.codehaus.org/browse/JRUBY-7201 )

@ayappanec
Copy link

@ayappanec ayappanec commented Jan 23, 2015

Can any maintainers look into this issue ( jnr/jffi#12 )

@headius
Copy link
Member

@headius headius commented Jan 23, 2015

Wow, apologies for missing this one for so long. The PR has been merged and we'll try to get releases of the whole JNR chain out for JRuby 1.7.x and 9k.pre2.

@headius headius closed this Jan 23, 2015
@headius headius added this to the 9.0.0.0.pre2 milestone Jan 23, 2015
@headius
Copy link
Member

@headius headius commented Jan 23, 2015

Oops...don't close until we have updated JNR-* in a JRuby release.

@headius headius reopened this Jan 23, 2015
@ayappanec
Copy link

@ayappanec ayappanec commented Jan 28, 2015

Can this be added to upcoming JRuby 1.7.x release also ?

@ayappanec
Copy link

@ayappanec ayappanec commented Feb 4, 2015

Any update for the above comment?

@gustavotemple
Copy link
Contributor

@gustavotemple gustavotemple commented Feb 25, 2015

@ayappanec, did you see this issue?

jnr/jffi#15

@headius
Copy link
Member

@headius headius commented Feb 25, 2015

I don't see why we couldn't update this in 1.7.x as well. @enebo and I know we need to update everything for next releases, which will involve fixing some regressions in the jnr-* chain of projects. Marking for 1.7.20.

@headius headius added this to the JRuby 1.7.20 milestone Feb 25, 2015
@headius headius removed this from the 9.0.0.0.pre2 milestone Feb 25, 2015
@headius
Copy link
Member

@headius headius commented Feb 26, 2015

jnr/jffi#16 details the now-mitigated showstopper preventing us from releaseing jffi 1.2.8. Should be able to get the family released this week.

@ayappanec
Copy link

@ayappanec ayappanec commented Mar 2, 2015

I came to know about the problem you guys facing for releasing jffi 1.2.8. I hope this one will get into Jruby 1.7.20 & 9000. Thank you.

@enebo
Copy link
Member

@enebo enebo commented Mar 3, 2015

@headius was able to release jffi and dependent artifacts yesterday so the plan is to update master and jruby-1_7 before our next releases.

@headius
Copy link
Member

@headius headius commented Mar 3, 2015

1.7 and master have both been updated for latest jnr-* projects.

If it's possible for someone on PLinux/ppc64 to do a build of jruby-1_7 branch and confirm it works properly, we'd appreciate it :-)

@headius headius closed this Mar 3, 2015
@ayappanec
Copy link

@ayappanec ayappanec commented Mar 4, 2015

Building jruby-1_7 branch fails in PLinux/ppc64 & PLinux/ppc64le due to unknown compilation issue.
"Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project jruby-core: Compilation failure"

But the main problem here is we are not able to build jruby-complete jar individually due to missing artfifacts.
"Failed to execute goal on project jruby-complete: Could not resolve dependencies for project org.jruby:jruby-complete:bundle:1.7.20-SNAPSHOT: The following artifacts could not be resolved: org.jruby:jruby-core:jar:noasm:1.7.20-SNAPSHOT, org.jruby:jruby-stdlib:jar:1.7.20-SNAPSHOT: Could not find artifact org.jruby:jruby-core:jar:noasm:1.7.20-SNAPSHOT in sonatype (https://oss.sonatype.org/content/repositories/snapshots/)"

It seems like the artifacts are not yet available in sonatype repository.

@mkristian
Copy link
Member

@mkristian mkristian commented Mar 4, 2015

to build jruby-complete you need to run from the basedir of jruby
mvn -Pcomplete
or mvn clean install -Pcomplete
if this is not passing beyond jruby-core then nothing furtehr down the road
will work since all those missing artifacts will be build on the way

On Wed, Mar 4, 2015 at 8:25 AM, ayappanec notifications@github.com wrote:

Building jruby-1_7 branch fails in PLinux/ppc64 & PLinux/ppc64le due to
unknown compilation issue.
"Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.1:compile
(default-compile) on project jruby-core: Compilation failure"

But the main problem here is we are not able to build jruby-complete jar
individually due to missing artfifacts.
"Failed to execute goal on project jruby-complete: Could not resolve
dependencies for project org.jruby:jruby-complete:bundle:1.7.20-SNAPSHOT:
The following artifacts could not be resolved:
org.jruby:jruby-core:jar:noasm:1.7.20-SNAPSHOT,
org.jruby:jruby-stdlib:jar:1.7.20-SNAPSHOT: Could not find artifact
org.jruby:jruby-core:jar:noasm:1.7.20-SNAPSHOT in sonatype (
https://oss.sonatype.org/content/repositories/snapshots/)"

It seems like the artifacts are not yet available in sonatype repository.


Reply to this email directly or view it on GitHub
#1389 (comment).

@ayappanec
Copy link

@ayappanec ayappanec commented Mar 4, 2015

I tried both the options but they failed to get beyond jruby-core. Finally i went into the maven/jruby-complete directory and tried to build jruby-complete jar by issuing "mvn clean install" and there i faced this missing artifacts issue..

@headius
Copy link
Member

@headius headius commented Mar 4, 2015

@ayappanec Please open a separate issue for your problems building JRuby on PLinux and we'll work through them separately. I suspect it's some oddity in how our build interacts with the JVM on that platform.

@ayappanec
Copy link

@ayappanec ayappanec commented Mar 4, 2015

@mkristian It requires jruby-core,jruby-stdlib and other such jars also.Can you please place those jars also in sonatype?
@headius Sure. I will open a separate issue for building JRuby on PLinux

@mkristian
Copy link
Member

@mkristian mkristian commented Mar 4, 2015

@ayappanec
Copy link

@ayappanec ayappanec commented Mar 4, 2015

I tried with the command mentioned above. its trying to download the jruby-core jar from sonatype but failed since the jar uploaded there has some number associated with the filename "jruby-stdlib-1.7.20-20150304.132127-1.jar"

@headius
Copy link
Member

@headius headius commented Mar 4, 2015

@ayappanec Thank you...please let us know when you've opened the other issue. For confirming this issue right now, just download the 1.7.20 build that @mkristian provided above.

@ayappanec
Copy link

@ayappanec ayappanec commented Mar 5, 2015

@headius I could see the jffi binaries for ppc64 & ppc64le in the jruby-complete jar. Thank you

@ayappanec
Copy link

@ayappanec ayappanec commented Mar 5, 2015

@headius I opened a new issue for Jruby build failure (#2648)

@ayappanec
Copy link

@ayappanec ayappanec commented Mar 31, 2015

It seems like we need ppc64le jnr-ffi TypeAliases and ruby ffi ppc64le platform specific types.conf as well to get the latest jruby working ppc64le. I could see the changes already went into the master branch of jnr-ffi & jruby. I don't know when the jnr-ffi next release is going to happen. If it happens before jruby release, then we need to change the jnr-ffi version to 2.0.3 in jruby to make use of it. Otherwise we have wait till further releases of jruby.

@headius
Copy link
Member

@headius headius commented Apr 6, 2015

@ayappanec I'm largely paralyzed from improving ppc64le support right now without your help, so if there are additional issues we need to fix, I'm hoping to see PRs from you :-)

@ayappanec
Copy link

@ayappanec ayappanec commented Apr 7, 2015

@headius Sure.. Right now #2790 & #1897 seems to be the major hurdle. #2647 is yet another one but we have openjdk as an option here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
8 participants