classpath 0.99 build issue #49

Closed
VVette opened this Issue Nov 6, 2013 · 71 comments

Projects

None yet

7 participants

@VVette
VVette commented Nov 6, 2013

classpath do_configure fails with following error :

configure:24669: ecj-initial -nowarn -source 1.5 -target 1.5 Object.java
Exception occurred while VM initialising.
java/lang/NullPointerException

Thank you for your help on this topic

@mindrunner
Contributor

same / similar problem here.

How can we start investigating this?

@VVette
VVette commented Nov 13, 2013

I am quite a newbie in linux development and bitbake use, so it is difficult for me to find the source of the problem.
I could'nt identify wether it comes from jamvm-initial or classpath-initial, if you check the configure and compile logs everything looks ok. Which gcc version do you use? I am on Ubuntu 13.10, gcc 4.8.1

@mindrunner
Contributor

Also using 4.8.1, but on Gentoo.

Pasted my error log:
http://pastebin.com/PX7KrC7d

@VVette
VVette commented Nov 13, 2013

The log is the same as mine. Looks like the problem comes form ecj-initial, which calls jamvm-initial.
Could not run "hello world" with this jamvm initial. Do you think it may come from jamvm or from classpath-initial (0.93)?

@mindrunner
Contributor

I thougt about missing dependencies, because there are many "No such file errors" (See line 499)

@VVette
VVette commented Nov 13, 2013

according to this thread http://www.linuxquestions.org/questions/linux-software-2/prelink-configuration-error-fatal-error-ac_nonexistent-h-no-such-file-or-direct-939408/ it may not be a problem.
But the fact that ecj-initial does not work is really an issue because classpath 0.99 will by compiled with this tool (as per my understanding)

@mindrunner
Contributor

Yeah youre right. I just saw that the problem is a segfaulting jamvm-initial

@VVette
VVette commented Nov 13, 2013

looks like your problem is not exactly the same as mine. In the config.log I do not get a segfault but this error :
configure:24669: ecj-initial -nowarn -source 1.5 -target 1.5 Object.java
Exception occurred while VM initialising.
java/lang/NullPointerException

@mindrunner
Contributor

Acutally this doesnt make it better! :D

I am playing around with bb files, but no luck, yet! :(

@VVette
VVette commented Nov 14, 2013

can you run a hello world program with your jamvm initial? if no, which message do you get?

@mindrunner
Contributor

I am getting a:

Aborting the VM -- couldn't allocate the heap: Cannot allocate memory

but dont know if I do it the right way. What steps do you follow to test it with a simple HelloWorld?

@VVette
VVette commented Nov 14, 2013

I found hello.jar here : http://code.google.com/p/kmttg/downloads/detail?name=hello.jar&can=4&q=

then : "path-to-jamvm-initial"/jamvm-initial -jar hello.jar
You do not have the same error as me. Did you have a look at this kind of thread (look at the "-Xmx512m" arg)? http://www.mail-archive.com/openembedded-devel@lists.openembedded.org/msg02555.html

@mindrunner
Contributor

jamvm-initial -Xmx512m -jar hello.jar ” terminated by signal SIGSEGV (Address boundary error)

@VVette
VVette commented Nov 14, 2013

ok, investigating in jamvm source code to find the function that fails

@VVette
VVette commented Nov 14, 2013

can you explain how you got this log? I never used execve and would be interested in doing the same test on my computer

@mindrunner
Contributor

see "man strace" ;)

I configured my yocto/oe environment to use my local crossdev arm toolchain, instead of the internal on. will check if this changes somethin. But since I am on battery now, I have to test it later.

@woglinde
Owner

Hi,

sorry for the late answer. Yes it is possible that newer gcc miscompiles jamvm-initial. I had no time yet to investigate it. It worked for me with ubuntu 10.12 at least. I could possible be true too, that I tested some patches
and therefore did no see this error.

@mindrunner
Contributor

Ubuntu 10.12 does not exist, 10.10, or 12.10 maybe?
I tried switching to 4.7 gcc series, but no luck.
I will now cherrypick meta-java overlay to find the breaking commit (if there is one)

@woglinde
Owner

Sorry my fault, 12.10 I meant.

@mindrunner
Contributor

I tired some random commits, going back to somewhere in 2012. Everytime the same error occures.
On which packeges in other overlays meta-java depends? Maybe we also have to investogate there?
(Currently building with gcc 4.7 series and uclibc, instead of eglibc, but I really think that this will result in the same problems)
Can anyone test on an outdated ubuntu? Like 10.10 or sth? On which kind of systen was this succesfully tested?

@mindrunner
Contributor

I think I made a step into the right direction. If I change the Compiler on my host system (not the GCCVERSION in TCMODE) to 4.7 the failing package is icedtea7-native_2.1.3.bb

Ill try with icedtea 6 now. Maybe this will work.
Also my build host pulled in gcc 4.8.2 recently. This one I will also check.

Reporting soon...

@mindrunner
Contributor

I have a working build with gcc4.7 hostcompiler AND GCCVERSION set to 4.7.

@mindrunner
Contributor

Maybe we should consider upgrading the packages to most recent version. I see that openjdk-7 packages are around 5 months old.

@woglinde what is you plan with that? How can I contact you? Maybe you can give me a short introsuction how to update the receipt. :))

@VVette
VVette commented Nov 22, 2013

hi, any news on the topic?
On my side, with Ubuntu 12.10 compilation the failing package is icedtea7-native_2.1.3.bb
http://pastebin.com/G4BxuYcZ
I feel so noob ;-) really no idea of what is happening right here!
Actually trying to compile openjdk6, ubuntu 12.10, will report soon
@mindrunner could you teach me how to set GCCVERSION in TCMODE (sorry if the question sounds stupid)

@VVette
VVette commented Nov 22, 2013

sorry for the silly questions but, how can you rebuild a package from scratch (do_configure...) after changing parameters?

@toolmmy
toolmmy commented Nov 22, 2013

Hi VVette,
cou can set the GCCVERSION in your local.conf:
GCCVERSION = "4.7"
SDKGCCVERSION = "4.7"

If you want to clean the current configuration of your recipe you only need to call 'bitbake -c clean '

I'm able to build the recipie on my machine/configuration:

  • yocto-poky-1.5
  • Ubuntu-12.04
  • target is arm-cortex-a8-hardfp

make sure you have enough resources on your machine. The build requires a lot of memory (and/or swap which makes the build pretty slow)!

sometimes it is a good idea to build icedtea7-native before building openjdk (if you have parallel builds):
bitbake -k icedtea7-native
bitbake -k openjdk7

@mindrunner
Contributor

GCCVERSION = "4.7%"
SDKGCCVERSION = "4.7%"

fits better ;)

(this will allow any 4.7 series gcc)

@woglinde
Owner

The problem here is not the resulting compiler in oe, but the host-compiler of your distribution.

To work around the failing icedtea7-native edit the openjdk7 recipe and force it to use icedtea6-native too.

@mindrunner
Contributor

This all really does not help. I am now on a dedicated-build-system. I cannot build that classpath package. Th only gcc installed is 4.7 series. What else could brake the build here?

@VVette
VVette commented Nov 25, 2013

@woglinde hello. I don't understand why both openjdk6 and 7 use icedtea7. If I look at openjdk6 bb files it seems that iced6 should be used. Could you tell us more about how iced version is managed in meta-java?
Thanks

@VVette
VVette commented Nov 25, 2013

still have the same miscompiling on icedtea7 whith gcc 4.7 hostcompiler and GCCVERSION set to 4.7% ... :-(

@mindrunner
Contributor

same problem here.

@woglinde
Owner

Hi again,

you need to edit recipes-core/openjdk/openjdk-7-common.inc and replace icedtea7-native with icedtea6-native in the DEPENDS-line.

Bye Henning

@mindrunner
Contributor

does not change anything here!

@mindrunner
Contributor

I just tried with a clean environment and as less changes than possible

. ./oe-init-build-env test
vi conf/bblayers.conf (added meta-java and meta-oe)
bitbake -k icedtea7-native

Loading cache: 100% |##########################################################################################################################################################################################################| ETA: 00:00:00
Loaded 1802 entries from dependency cache.
NOTE: Resolving any missing task queue dependencies
NOTE: multiple providers are available for jpeg-native (jpeg-native, libjpeg-turbo-native)
NOTE: consider defining a PREFERRED_PROVIDER entry to match jpeg-native

Build Configuration:
BB_VERSION = "1.21.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Gentoo"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "1.5+snapshot-20131128"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-yocto
meta-yocto-bsp = "master:2dc0c16821d08811abe6d9b4d46d942740700b58"
meta-oe = "master:9b3e284c58bce673801ca3649b53bc53797ce2e9"
meta-java = "master:14d6ad1608c66b8970d4052dce4cd1fc64fb6ea2"

Failing with the same error on classpath_native :(

@mindrunner
Contributor

Tried on a Debian wheezy. Same problem there...

@mindrunner
Contributor

I tried again... In an Ubuntu 12.04 docker image. So my resume is, that we have some ubuntu 12.04 working and some ubuntu 12.04 failing with segfaulting jamvm. Also there are gentoo systems with (nearly) the same configuration working, and some failing. (At least I only have one gentoo which compiles this fine.) Since I figured out, that bitbake (or the receipes) do not really resolve all needed dependencies, nor make an suitable error checking, I assume, that on the failing nodes, there is one or more packages missing. But I have no clue which one! :( Anyone out there with a hint?

@toolmmy
toolmmy commented Dec 12, 2013

jamvm-initial seems to break for the following reason:

jamvm-initial -verbose -cp /workspace/linux/yocto-poky-1.5/build/hbtouch-imx6/tmp/sysroots/x86_64-linux/usr/share/java/ecj-bootstrap.jar org.eclipse.jdt.internal.compiler.batch.Main
[Loaded java/lang/Object from /workspace/linux/yocto-poky-1.5/build/hbtouch-imx6/tmp/sysroots/x86_64-linux/usr/share/classpath-initial/glibj.zip]
[Linking class java/lang/Object]
[Loaded java/io/Serializable from /workspace/linux/yocto-poky-1.5/build/hbtouch-imx6/tmp/sysroots/x86_64
[...]
[Loaded gnu/java/util/regex/REMatch from /workspace/linux/yocto-poky-1.5/build/hbtouch-imx6/tmp/sysroots/x86_64-linux/usr/share/classpath-initial/glibj.zip]
[Linking class gnu/java/util/regex/REMatch]
Segmentation fault

I'm still investigating.
Environment:
Build Configuration:
BB_VERSION = "1.21.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "LinuxMint-16"
TARGET_SYS = "arm-poky-linux-gnueabi"
MACHINE = "imx6dlsabresd"
DISTRO = "poky"
DISTRO_VERSION = "1.5+snapshot-20131212"
TUNE_FEATURES = "armv7a vfp neon callconvention-hard cortexa9"
TARGET_FPU = "vfp-neon"
meta
meta-yocto
meta-yocto-bsp
meta-oe
meta-gnome = "master:ac3a5d430139e3be08718770e4439032ad3091eb"
meta-java = "master:14d6ad1608c66b8970d4052dce4cd1fc64fb6ea2"

@mindrunner
Contributor

Anything I can do to help you?

@toolmmy
toolmmy commented Dec 12, 2013

Can you please verify if you have the same error? You only need to build classpath native (bitbake classpath-natve) and wait for the do_configure error. After that you can go into your x86_64 sysroot ([...]/tmp/sysroots/x86_64-linux/usr/bin) and add it to your PATH var. Running ecj-initial should show the follow output:

/workspace/hbtouch/svn/branches/dot2wa2/linux/yocto-poky-1.5/build/hbtouch-imx6/tmp/sysroots/x86_64-linux/usr/bin $ ecj-initial
Running JamVM-initial: -Xmx512m -cp /workspace/linux/yocto-poky-1.5/build/hbtouch-imx6/tmp/sysroots/x86_64-linux/usr/share/java/ecj-bootstrap.jar org.eclipse.jdt.internal.compiler.batch.Main
Segmentation fault

If you start jamvm-initial in verbose mode, you should see the class loader:
jamvm-initial -verbose -Xmx512m -cp /workspace/linux/yocto-poky-1.5/build/hbtouch-imx6/tmp/sysroots/x86_64-linux/usr/share/java/ecj-bootstrap.jar org.eclipse.jdt.internal.compiler.batch.Main

@toolmmy
toolmmy commented Dec 12, 2013

I'm guessing the problem is jamvm-initial. So a very ugly workaround would be using the opendjk of your host machine instead of ecj-initial/jamvm-initial

@mindrunner
Contributor

on my hosts, jamvm-initial is the segfaulting binary. how to switch to the hosts jdk? i really wonder about why on one of my boxes the compilation succeedes...

@toolmmy
toolmmy commented Dec 12, 2013

I'm still investigating/looking for workarounds. I've tried to reproduce this error on a Ubuntu 12.04. and the do_configuration went through. It seems to me that everything > Ubuntu 12.04 has this issue. Can you please try Ubuntu 12.04 to verify this assumption?

@mindrunner
Contributor

No, I cannot confirm this. I have the same problem with ubuntu 12.04

@toolmmy
toolmmy commented Dec 13, 2013

I found a workaround that avoids the usage of jamvm during the initial configuration. It is only used during the initialization phase (do_configure). As a workaround I'm using the java version of my host machine (apt-get install openjdk-7-jdk). In order to do that you need to replace the following lines in the build/tmp/sysroots/x86_64-linux/usr/bin/java-initial script (this can be also done in a bitbake append file of the jamvm-initial recipe)

replace the line: jamvm-initial ${1+"$@"} with
"/usr/lib/jvm/default-java/bin/java ${1+"$@"}"

@mindrunner
Contributor

Do you already wrote a bbappend file? Ill test it later on today!

@mindrunner
Contributor

Getting another error now:

http://pastebin.com/vK1fnX6L

@woglinde
Owner

As the error says install libXtst with bitbake libXtst.

@mindrunner
Contributor

Compiles fine now, but didnt have time for testing my images.Will report then.

@toolmmy
toolmmy commented Dec 16, 2013

Great!

I dont get the segfault on a fresh version of ubuntu 12.04.3 (without any updates)

@woglinde: does it make sense to add a fallback in the java-initial that uses the java version of the host pc? What do you think?

@jackmitch

I can confirm that I am also having this issue, I would prefer not to have to munge files in the tmp directory, as it breaks the deterministic nature of a build...

log.do_configure http://ix.io/9qO

@toolmmy
toolmmy commented Dec 18, 2013

Hi Jack,

I'm not saying that we should go ahead with this workaround (changing the java-init by hand), I just wanted to know whether jamvm is the root cause or not. So we need to find a real solution for the segfault issue of jamvm. I had a brief look on jamvm and i realized that the meta-java layer is using a pretty old version of jamvm (1.4.5). I'm going to use the current version of jamvm (1.5.4) which might work. I will keep you in touch.

@mindrunner
Contributor

Would be worth a try! :D There are some other outdated packages I think. Maybe we should analyze and keep them up to date. I am thinking of using this for production in future. So better have no java exploits in my image! ;)

@jackmitch

So I have tried bumping the jamvm version and I start hitting problems with not being able to find classes in classpath:

| configure:24760: /mnt/seaRaid/Projects/oe-core.git/beaglebone-ben/tmp-eglibc/sysroots/x86_64-linux/usr/bin/ecj-initial -nowarn -source 1.5 -target 1.5 Object.java
| Exception occurred while VM initialising.
| java/lang/NoClassDefFoundError: java/lang/Class
| configure:24763: $? = 1
| configure:24767: error: The Java compiler /mnt/seaRaid/Projects/oe-core.git/beaglebone-ben/tmp-eglibc/sysroots/x86_64-linux/usr/bin/ecj-initial failed (see config.log, check the CLASSPATH?)

If there are any suggestions on how I might fix this I would be open to suggestions. If someone wants me to punt my changes over the wall I can put them up.

@toolmmy
toolmmy commented Dec 19, 2013

you need to specify the bootclasspath. this can be done in a do_configure_prepand

export BOOTCLASSPATH = "${STAGING_DATADIR_NATIVE}/jamvm-initial/classes.zip:${STAGING_DATADIR_NATIVE}/classpath-initial/glibj.zip"

@jackmitch

That seemed to get me a bit further but now the invocation of the compile just fails.

config.log http://ix.io/9sk
log.do_configure http://ix.io/9sl

@toolmmy
toolmmy commented Dec 19, 2013

could you please provide your changes in a separate fork?

Update:
I did some research on the build process with the following result:
do_configure of classpath-native is using ecj-initial to run the ecj-stuff. In order to run the ecj-bootstrap java-initial is used. Java-initial itself is wrapper for jamvm which restarts the jamvm if it returns with a segfault. It seams to me that this segfaulting behavior is well known.

(see https://github.com/woglinde/meta-java/blob/master/recipes-core/jamvm/files/java-initial)

@jackmitch

https://github.com/jackmitch/meta-java/tree/updates

it's preliminary, and rough so bear that in mind.

@toolmmy
toolmmy commented Dec 19, 2013

Thanks. Looks pretty similar to my attempts. I have just one suggestions; be careful with mixing jamvm-initial and jamvm-native. Currently both will be installed in the same folder (usr/share/jamvm).

I have also commited my changes (toolmmy/meta-java@b282279) . Bootstrap looks fine, but I'm getting the following error (at least no segfault):

http://pastebin.com/EquAkXaa (configure:24760)

@woglinde
Owner

Hm intressting that jamvm 1.5 can be compiled with jikes. I thought it would not work, but I never tried it.

The processs was to build slowly up the apis:

  1. compile jikes for getting java 1.4, which can compile classpath-initial
  2. with jikes and classpath-intial we can build jamvm-initial 1.4 which provides java-runtime 1.4
  3. with jamvm-initial we can build ecj-boostrap which provides a javac 1.5 compiler
  4. than using ecj-boostrap and jamvm-initial and classpasth-initial to compile classpath-native and ecj-initial
  5. than with classpath-native and ecj-initial we build jamvm-native
  6. at this point we could build openjdk/icedtea with jamvm and classpath-native for the tagret, unfornatly there were some problems with jamvm and cacao which gave random errors and so we build icedtea/openjdk-native
  7. with icedtea-native we build the target openjdk/icedtea which is alot easier than some years before

PS: I hope I will find some time during christmas holidays to put meta-java back in a reliable state.

@mindrunner
Contributor

Nice to see a progress here. I hope we can get this in a working state, so that java makes fun again with openembedded! :)

@sledz
Contributor
sledz commented Jan 8, 2014

Since upgrading my build host from openSUSE 12.3 to 13.1 i hit the same problem. So for us this is really a critical issue.

@jackmitch

There is a patch posted to the oe-dev list which relates to meta-java, it touchest classpath but I don't think it will solve the problem. Is maybe worth merging anyway though.

https://www.mail-archive.com/openembedded-devel@lists.openembedded.org/msg34182.html

@woglinde
Owner
woglinde commented Jan 8, 2014

Hi,

I have worked on it over the holidays and will commit something this evening, fornatly cacao 0.98 works after some tweaking and can replace jamvm-initial, which needs more work to get it working again.

@woglinde
Owner
woglinde commented Jan 9, 2014

Stuff is pushed, so I am looking forward to close the issue soon.

@toolmmy
toolmmy commented Jan 9, 2014

Hi woglinde,
your patches seems to work! thanks alot for your support - I really do appreciate that!

@mindrunner
Contributor

Looks better now and is working for me!

@sledz sledz added a commit to DFE/HIPOS that referenced this issue Jan 14, 2014
@sledz sledz update meta-java submodule
This fixes a classpath-0.99 build issue on build hosts with newer
linux distributions.

see woglinde/meta-java#49

Signed-off-by: Steffen Sledz <sledz@dresearch-fe.de>
5473bce
@woglinde
Owner

problem is fixed for now

@woglinde woglinde closed this Jan 14, 2014
@jianxinren

can someone point me to woglinde's patch? I am having similar issue now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment