Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

jruby core build fails on IBM ppc64 #2647

Closed
ralphbellofatto opened this Issue Mar 4, 2015 · 26 comments

Comments

Projects
None yet
7 participants
@ralphbellofatto
Copy link

ralphbellofatto commented Mar 4, 2015

We are attempting to build the tip of the git branch of jruby 1.7 to test out a recent fix.

When we execute: mvn -Pcomplete

The system reports

[INFO] JRuby Core ......................................... FAILURE [  6.385 s]

there is no more error messages in the output:

we are attempting this on a power 7 (ppc64) architecture computer.

here is the output of the mvn --version command

[ralphbel@bgxcat jruby-jruby-1_7]$ mvn --version
Apache Maven 3.2.5 (12a6b3acb947671f09b81f49094c53f426d8cea1; 2014-12-14T12:29:23-05:00)
Maven home: /opt/apache-maven-3.2.5
Java version: 1.7.0, vendor: IBM Corporation
Java home: /opt/ibm/java-ppc64-71/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "2.6.32-431.23.3.el6.ppc64", arch: "ppc64", family: "unix"

here is the complete maven output.

[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO] 
[INFO] JRuby
[INFO] JRuby Core
[INFO] JRuby Ext
[INFO] JRuby Readline
[INFO] JRuby Ripper
[INFO] JRuby Lib Setup
[INFO] JRuby Artifacts
[INFO] JRuby Stdlib
[INFO] JRuby Complete
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building JRuby 1.7.20-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-enforcer-plugin:1.0:enforce (enforce-maven) @ jruby-parent ---
[INFO] 
[INFO] --- maven-install-plugin:2.4:install (default-install) @ jruby-parent ---
[INFO] Installing /home/ralphbel/jruby/jruby-jruby-1_7/pom.xml to /gsa/yktgsa/home/r/a/ralphbel/.m2/repository/org/jruby/jruby-parent/1.7.20-SNAPSHOT/jruby-parent-1.7.20-SNAPSHOT.pom
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building JRuby Core 1.7.20-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-enforcer-plugin:1.0:enforce (enforce-maven) @ jruby-core ---
[INFO] 
[INFO] --- properties-maven-plugin:1.0-alpha-2:read-project-properties (properties) @ jruby-core ---
[WARNING] Ignoring missing properties file: /home/ralphbel/jruby/jruby-jruby-1_7/core/../build.properties
[INFO] 
[INFO] --- buildnumber-maven-plugin:1.2:create (jruby-revision) @ jruby-core ---
[INFO] Checking for local modifications: skipped.
[INFO] Updating project files from SCM: skipped.
[INFO] ShortRevision tag detected. The value is '7'.
[INFO] Executing: /bin/sh -c cd /home/ralphbel/jruby/jruby-jruby-1_7/core && git rev-parse --verify --short=7 HEAD
[INFO] Working directory: /home/ralphbel/jruby/jruby-jruby-1_7/core
[INFO] Storing buildNumber: null at timestamp: 1425425250957
[INFO] ShortRevision tag detected. The value is '7'.
[INFO] Executing: /bin/sh -c cd /home/ralphbel/jruby/jruby-jruby-1_7/core && git rev-parse --verify --short=7 HEAD
[INFO] Working directory: /home/ralphbel/jruby/jruby-jruby-1_7/core
[INFO] Storing buildScmBranch: UNKNOWN_BRANCH
[INFO] 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ jruby-core ---
[INFO] Using 'utf-8' encoding to copy filtered resources.
[INFO] Copying 44 resources
[INFO] Copying 1 resource
[INFO] Copying 1 resource to /home/ralphbel/jruby/jruby-jruby-1_7/core/src/main/java
[INFO] 
[INFO] --- maven-compiler-plugin:3.1:compile (anno) @ jruby-core ---
[INFO] Compiling 7 source files to /home/ralphbel/jruby/jruby-jruby-1_7/core/target/classes
[INFO] 
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ jruby-core ---
[INFO] Compiling 1356 source files to /home/ralphbel/jruby/jruby-jruby-1_7/core/target/classes
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] JRuby .............................................. SUCCESS [  1.168 s]
[INFO] JRuby Core ......................................... FAILURE [  6.385 s]
[INFO] JRuby Ext .......................................... SKIPPED
[INFO] JRuby Readline ..................................... SKIPPED
[INFO] JRuby Ripper ....................................... SKIPPED
[INFO] JRuby Lib Setup .................................... SKIPPED
[INFO] JRuby Artifacts .................................... SKIPPED
[INFO] JRuby Stdlib ....................................... SKIPPED
[INFO] JRuby Complete ..................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 10.252 s
[INFO] Finished at: 2015-03-03T18:27:34-05:00
[INFO] Final Memory: 15M/24M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project jruby-core: Compilation failure -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :jruby-core
@mkristian

This comment has been minimized.

Copy link
Member

mkristian commented Mar 4, 2015

the missing error message comes from "fork=true" when we compile the code which is needed to add the unsafe.jar to bootloader. with fork=false things compile but 2-3 classes using this unsafe classes.

seeing a IBM jdk it looks like the unsafe classes are probably the problem - just guessing here.

@ralphbellofatto

This comment has been minimized.

Copy link
Author

ralphbellofatto commented Mar 4, 2015

@mkristian

This comment has been minimized.

@ralphbellofatto

This comment has been minimized.

Copy link
Author

ralphbellofatto commented Mar 4, 2015

after changing core/pom fork from true to false, we now see some errors:

[ERROR] /home/ralphbel/jruby/jruby-jruby-1_7/core/src/main/java/org/jruby/util/unsafe/UnsafeHolder.java:[96,10]
cannot find symbol
symbol: method fullFence()
location: variable U of type sun.misc.Unsafe
[ERROR] /home/ralphbel/jruby/jruby-jruby-1_7/core/src/main/java/org/jruby/util/unsafe/UnsafeHolder.java:[100,10]
cannot find symbol
symbol: method loadFence()
location: variable U of type sun.misc.Unsafe
[ERROR] /home/ralphbel/jruby/jruby-jruby-1_7/core/src/main/java/org/jruby/util/unsafe/UnsafeHolder.java:[104,10]
cannot find symbol
symbol: method storeFence()
location: variable U of type sun.misc.Unsafe

So the question here is how do we fixee when building this on ppc64
machines...

@headius

This comment has been minimized.

Copy link
Member

headius commented Mar 12, 2015

So I suspect the problem is that the IBM JDK is not honoring boot classloader jars.

For these Java 8+ Unsafe "fence" methods, we use a library called unsafe-mock (https://github.com/headius/unsafe-mock) to compile against. The build needs to be able to put those jars in the boot classpath of compile so it sees our Unsafe instead of whatever the current JDK provides.

If you pass -X, somewhere in the reams of output you should see a javac command line (or everything on the command line except "javac"). Can you get that output and try running it directly (i.e. not through maven)?

@headius headius changed the title jruby core fails with no error message jruby core build fails on IBM ppc64 Mar 12, 2015

@Saganesque

This comment has been minimized.

Copy link

Saganesque commented Mar 19, 2015

I'm running into this trying to install jruby via rvm on a vagrant box running Ubuntu 14.04. Error runnin mvn: Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project jruby-core: Compilation failure -> [Help 1]

If i want to continue to use this tech stack, what do I need to do to make the install complete?

@mkristian

This comment has been minimized.

Copy link
Member

mkristian commented Mar 19, 2015

@Saganesque just today I could reproduce the same error when I used ibm jdk to compile jruby on my amd64 linux. are you using a ibm jdk ?

@ralphbellofatto

This comment has been minimized.

Copy link
Author

ralphbellofatto commented Mar 19, 2015

my failure telemetry indicates IBM java on this specific failure.

@Saganesque

This comment has been minimized.

Copy link

Saganesque commented Mar 19, 2015

@mkristian sure am.

@Saganesque

This comment has been minimized.

Copy link

Saganesque commented Mar 19, 2015

@mkristian On a related note, when I compiled jruby 1.6.8 just now, I get the following:

/home/vagrant/.rvm/src/jruby-1.6.8/build.xml:236: Unable to find a javac compiler;
com.sun.tools.javac.Main is not on the classpath.
Perhaps JAVA_HOME does not point to the JDK.
It is currently set to "/usr/lib/jvm/java-7-openjdk-amd64/jre"

It's difficult to overstate my noobery. I'm unsure what I should be looking for there. There is a jdk install in that locale. Is there a remedy perhaps by switching thigs up with a different/older jdk install?

@Saganesque

This comment has been minimized.

Copy link

Saganesque commented Mar 19, 2015

@mkristian boned it. I have open jdk

@Saganesque

This comment has been minimized.

Copy link

Saganesque commented Mar 19, 2015

And after jdk uninstall then new oracle 8 jdk install, same problem for jruby-head :(

@mkristian

This comment has been minimized.

Copy link
Member

mkristian commented Mar 20, 2015

@Saganesque just checked it from my side without rvm and fairly recent ubuntu installation:
$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=14.10
DISTRIB_CODENAME=utopic
DISTRIB_DESCRIPTION="Ubuntu 14.10"

$ java -version
java version "1.8.0_40"
Java(TM) SE Runtime Environment (build 1.8.0_40-b25)
Java HotSpot(TM) 64-Bit Server VM (build 25.40-b25, mixed mode)

and I took the current head from github. everything works with openjdk and oracle-jdk - the ibm-jdk has some problems with those "unsafe". I suspect this a RVM problem (not on machine where I have it installed right now) - you could see if you can compile the current master of jruby github

I know rbenv uses bin archives for installing jruby, i.e. it does not compile anything when installing jruby - maybe that is an option (just thinking).

@ralphbellofatto

This comment has been minimized.

Copy link
Author

ralphbellofatto commented Mar 20, 2015

Are you building on ppc64 (big endian) or a ppc64le machine (little endian)?

I'm building on a Redhat 7 ppc64 (bigendian) machine.

@Saganesque

This comment has been minimized.

Copy link

Saganesque commented Mar 23, 2015

Sorry for delay, been out with surgery on arm.
Fixed my issue by starting with a 12.04LTS 64-bit PC (AMD64) desktop vagrant box and provisioning with rvm. Same ops worked fine. Haven't tried 14.04 upgrade path yet.

@ayappanec

This comment has been minimized.

Copy link

ayappanec commented Mar 23, 2015

@headius I captured the output command line options and run it directly using javac

javac -d /home/ayappan/jruby/core/target/classes -classpath /home/ayappan/jruby/core/target/classes:/home/ayappan/.m2/repository/org/ow2m2/repository/org/ow2/asm/asm-commons/5.0.3/asm-commons-5.0.3.jar:/home/ayappan/.m2/repository/org/ow2/asm/asm-tree/5.0.3/asm-tree-5.0.3.jar:/home/ayappan/.m2/repoalysis-5.0.3.jar:/home/ayappan/.m2/repository/org/..................
.......................

......................

I got the same errors

[checking org.jruby.util.unsafe.UnsafeHolder]
/home/ayappan/jruby/core/src/main/java/org/jruby/util/unsafe/UnsafeHolder.java:96: error: cannot find symbol
U.fullFence();
^
symbol: method fullFence()
location: variable U of type Unsafe
/home/ayappan/jruby/core/src/main/java/org/jruby/util/unsafe/UnsafeHolder.java💯 error: cannot find symbol
U.loadFence();
^
symbol: method loadFence()
location: variable U of type Unsafe
/home/ayappan/jruby/core/src/main/java/org/jruby/util/unsafe/UnsafeHolder.java:104: error: cannot find symbol
U.storeFence();
^
symbol: method storeFence()
location: variable U of type Unsafe

We are using Java 7 version
[ayappan@soe07-vm4 jruby]$ mvn -version
Apache Maven 3.2.1 (ea8b2b07643dbb1b84b6d16e1f08391b666bc1e9; 2014-02-14T11:37:52-06:00)
Maven home: /home/ayappan/apache-maven-3.2.1
Java version: 1.7.0, vendor: IBM Corporation
Java home: /home/ayappan/ibm-java-ppc64le-71/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "3.10.0-229.ael7b.ppc64le", arch: "ppc64le", family: "unix"

@ralphbellofatto

This comment has been minimized.

Copy link
Author

ralphbellofatto commented Mar 23, 2015

I wonder if there is a requirement to build with jdk 8 for jruby...

the fullFenc, loadFence and storeFence are not part of the jdk 7 standard.

Here is the discussion for including them in Jdk 8: http://openjdk.java.net/jeps/171

@headius

This comment has been minimized.

Copy link
Member

headius commented Apr 6, 2015

The IBM javac does not appear to honor boot classpath during build, which is necessary for us to build using our fake Unsafe. This is the source of those "fence" API compilation errors on IBM JDK 7.

If you can't build with a Java 8 version of IBM's JDK, we'll need to figure out how to get JDK 7 to honor our mock-unsafe library during the build.

@tdaitx

This comment has been minimized.

Copy link
Contributor

tdaitx commented Apr 22, 2015

Be careful with the fork argument of the maven-compiler-plugin. The error you are seeing by setting it to false might not be the same as with it set to true.

With fork=false:

  1. compilerArgs will be ignored
  2. the internal maven compiler will be used, thus if running on JDK 7 then it will fail for sure since the unsafe-mock.jar was never set in classbootstrap (compilerArgs being ignored)

With fork=true:

  1. compilerArgs is enabled and set
  2. the executable set in the executable parameter of the maven-compiler-plugin is called, by default this is set simply to javac, thus the first one in $PATH will be called

So even with fork=true the wrong javac might be called unless:

  1. The right javac binary is included in the PATH - eg. if using JAVA_HOME simply run export PATH=$JAVA_HOME/bin:$PATH (portable, but requires this to be documented on the project)
  2. The executable parameter of the maven-compiler-plugin is set to the full path of the right javac to be called (non-portable)

My bet is that the PATH is not properly set and the wrong javac is being called (gcj maybe?), thus causing the Compilation failure error seen on the original report.

The comments referring to (or relying) on fork=false should be ignored since that is an expected error on JDK 7.

@tdaitx

This comment has been minimized.

Copy link
Contributor

tdaitx commented Apr 23, 2015

The problem was that IBM Java does not like to have the unsafe-mock.jar prepending the rt.jar in bootclasspath on runtime.

$ /opt/ibm/java-ppc64le-71/bin/javac -J-Xbootclasspath/p:unsafe-mock-8.0.jar UnsafeHolder.java
Exception in thread "main" java.lang.NoSuchMethodError: sun/misc/Unsafe.allocateDBBMemory(J)J
        at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:132)
        at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:306)
        at sun.misc.Perf.createLong(Native Method)
        at sun.misc.PerfCounter.<init>(PerfCounter.java:78)
        at sun.misc.PerfCounter.newPerfCounter(PerfCounter.java:84)
        at sun.misc.PerfCounter$CoreCounters.<clinit>(PerfCounter.java:141)
        at sun.misc.PerfCounter.getZipFileOpenTime(PerfCounter.java:195)
        at java.util.zip.ZipFile.<init>(ZipFile.java:232)
        at java.util.zip.ZipFile.<init>(ZipFile.java:161)
        at java.util.jar.JarFile.<init>(JarFile.java:169)
        at java.util.jar.JarFile.<init>(JarFile.java:106)
        at sun.misc.URLClassPath$JarLoader.getJarFile(URLClassPath.java:999)
        at sun.misc.URLClassPath$JarLoader.access$700(URLClassPath.java:849)
        at sun.misc.URLClassPath$JarLoader$1.run(URLClassPath.java:934)
        at sun.misc.URLClassPath$JarLoader$1.run(URLClassPath.java:924)
        at java.security.AccessController.doPrivileged(AccessController.java:333)
        at sun.misc.URLClassPath$JarLoader.ensureOpen(URLClassPath.java:923)
        at sun.misc.URLClassPath$JarLoader.<init>(URLClassPath.java:896)
        at sun.misc.URLClassPath$3.rtJarLoader(URLClassPath.java:619)
        at sun.misc.URLClassPath$3.run(URLClassPath.java:569)
        at sun.misc.URLClassPath$3.run(URLClassPath.java:559)
        at java.security.AccessController.doPrivileged(AccessController.java:333)
        at sun.misc.URLClassPath.getLoader(URLClassPath.java:558)
        at sun.misc.URLClassPath.getLoader(URLClassPath.java:521)
        at sun.misc.URLClassPath.getResource(URLClassPath.java:331)
        at java.net.URLClassLoader$ClassFinder.run(URLClassLoader.java:1224)
        at java.security.AccessController.doPrivileged(AccessController.java:369)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:661)
        at java.lang.ClassLoader.loadClassHelper(ClassLoader.java:942)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:869)
        at java.lang.ClassLoader.loadClassHelper(ClassLoader.java:934)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:869)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:336)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:847)
        at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:495)

Removing the -J argument fixes that during compilation:

$ /opt/ibm/java-ppc64le-71/bin/javac -Xbootclasspath/p:unsafe-mock-8.0.jar UnsafeHolder.java
@headius

This comment has been minimized.

Copy link
Member

headius commented Apr 23, 2015

So we can just remove it from the compiler plugin config?

@tdaitx

This comment has been minimized.

Copy link
Contributor

tdaitx commented Apr 23, 2015

OpenJDK 7 also works without -J, so it should be safe to remove it.

@lbianc

This comment has been minimized.

Copy link
Contributor

lbianc commented Apr 23, 2015

I built and run the tests on ppc64le and x86_64 without the -J and it worked.

@headius

This comment has been minimized.

Copy link
Member

headius commented Apr 23, 2015

Easy one!

@headius

This comment has been minimized.

Copy link
Member

headius commented Apr 23, 2015

Fixed on jruby-1_7 and master by 8b22c0b.

@headius headius closed this Apr 23, 2015

@mkristian

This comment has been minimized.

Copy link
Member

mkristian commented Apr 23, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.