Java running issues (Consolidated) #541

Closed
gameldar opened this Issue Jun 16, 2016 · 21 comments

Comments

Projects
None yet
@gameldar

I'm starting a new issue because I've seen we have a lot of issues now that are related to java running (e.g. #196, #253, and the ones relating to running Java 8 - after update 05?)

I think the causes for these may be different under water - but perhaps they are related and it would be easier to find them in one place.

In the cases I've reproduced the strace when running java will end up "hanging" with the following:
futex(0x7f06af8209d0, FUTEX_WAIT, 4646, NULL

This means the main thread is waiting on another thread - to see the full list of threads you can do a
ps -efL | grep java

Then you can attach strace to the individual pid of the thread (the third number in the list) by doing:
strace -p <pid>
There might be multiple threads that you need to search through to find the one that is causing the issue.

See https://meenakshi02.wordpress.com/2011/02/02/strace-hanging-at-futex/ for more information about using strace like this.

For Java 8 update 91. java -version this ends up with a thread with a repeating trace of the following:
--- SIGSEGV {si_signo=SIGSEGV, si_code=SI_KERNEL, si_addr=0} ---
rt_sigreturn()

For Gradle 2.14 on Oracle Java 1.7.0_80 this ends up with a repeated trace of the following:
--- SIGSEGV {si_signo=SIGSEGV, si_code=SI_KERNEL, si_addr=0x8} ---
gettimeofday({1466064257, 853996}, NULL) = 0
rt_sigreturn()

For ant 1.9.4 XSLT task on Oracle Java 1.7.0_80 has a very similar (si_addr is different) repeating trace of the following:
--- SIGSEGV {si_signo=SIGSEGV, si_code=SI_KERNEL, si_addr=0xc} ---
gettimeofday({1466065674, 398849}, NULL) = 0
rt_sigreturn()

@mphaney

This comment has been minimized.

Show comment
Hide comment
@mphaney

mphaney Jun 22, 2016

Thank you for this excellent walkthrough! Just tried this out again on build 14371 with Java 8 update 92 and produced the exact same SIGSEGV error. @benhillis, does this shed any light on things? REALLY anxious to get this up and working :-D

mphaney commented Jun 22, 2016

Thank you for this excellent walkthrough! Just tried this out again on build 14371 with Java 8 update 92 and produced the exact same SIGSEGV error. @benhillis, does this shed any light on things? REALLY anxious to get this up and working :-D

@therealkenc

This comment has been minimized.

Show comment
Hide comment
@therealkenc

therealkenc Jun 23, 2016

Collaborator

Even non-Java developers can get dragged into the long tail of Java because of (for example) Closure.

[edit] I thought I had a small gist demonstrating failure, but it was failing only due to a brain fart on my part; apologies. I'll keep trying to narrow it down.

Collaborator

therealkenc commented Jun 23, 2016

Even non-Java developers can get dragged into the long tail of Java because of (for example) Closure.

[edit] I thought I had a small gist demonstrating failure, but it was failing only due to a brain fart on my part; apologies. I'll keep trying to narrow it down.

@leninalive

This comment has been minimized.

Show comment
Hide comment
@leninalive

leninalive Jun 28, 2016

Dear guys, you are now my personal heroes: I have just spent more than 24 hours of continous troubleshooting (brrrr) to just figure out that adding -Xint to java command line is pretty enough workaround allowing to run any JDK/JRE you want (just tested it with latest Oracle JDK9 running Closure Compiler performing ADVANCED_OPTIMIZATIONS on jQuery) until you, Microsoft, fix those stupid bugs in shared memory related code in WSL. Thanks, I have no idea what would I do without you. And I also thinkthat it is the best time to go open source, really. Meanwhile, it interprets pretty fast.

leninalive commented Jun 28, 2016

Dear guys, you are now my personal heroes: I have just spent more than 24 hours of continous troubleshooting (brrrr) to just figure out that adding -Xint to java command line is pretty enough workaround allowing to run any JDK/JRE you want (just tested it with latest Oracle JDK9 running Closure Compiler performing ADVANCED_OPTIMIZATIONS on jQuery) until you, Microsoft, fix those stupid bugs in shared memory related code in WSL. Thanks, I have no idea what would I do without you. And I also thinkthat it is the best time to go open source, really. Meanwhile, it interprets pretty fast.

@therealkenc

This comment has been minimized.

Show comment
Hide comment
@therealkenc

therealkenc Jun 29, 2016

Collaborator

Someone should add a workaround-available tag to this issue. export _JAVA_OPTIONS="-Xint" is the ticket until a fix for whatever pattern JRE JIT uses is found.

Collaborator

therealkenc commented Jun 29, 2016

Someone should add a workaround-available tag to this issue. export _JAVA_OPTIONS="-Xint" is the ticket until a fix for whatever pattern JRE JIT uses is found.

@leninalive

This comment has been minimized.

Show comment
Hide comment
@leninalive

leninalive Jun 29, 2016

As I stated before, real problem is not in the JRE itself, but in the fact JIT compiler stores bytecode in shared memory. For some reason, sometimes it breaks. The strangest thing is that is heavily depends on what is happenning on Windows host. For example, there is some strange thing in Windows 10 called Unistack Service Group. As far as I found it looks like it handles all user account/sync information - I mean contacts/mail/skype uwp/messaging etc. Another strange thing there is that these services are all masked and not visible in Services - like some 'official' Microsoft malware. I has found that turning all these services off (leaving myself without synchronization and, probably, Microsoft without my telemetry) surprisingly improves situation with shared memory in WSL. At all, I must say that developers of that Unistack Service Group wrote it completely bad, it accounts for a lot of issues happenning in Windows 10.

leninalive commented Jun 29, 2016

As I stated before, real problem is not in the JRE itself, but in the fact JIT compiler stores bytecode in shared memory. For some reason, sometimes it breaks. The strangest thing is that is heavily depends on what is happenning on Windows host. For example, there is some strange thing in Windows 10 called Unistack Service Group. As far as I found it looks like it handles all user account/sync information - I mean contacts/mail/skype uwp/messaging etc. Another strange thing there is that these services are all masked and not visible in Services - like some 'official' Microsoft malware. I has found that turning all these services off (leaving myself without synchronization and, probably, Microsoft without my telemetry) surprisingly improves situation with shared memory in WSL. At all, I must say that developers of that Unistack Service Group wrote it completely bad, it accounts for a lot of issues happenning in Windows 10.

@leninalive

This comment has been minimized.

Show comment
Hide comment
@leninalive

leninalive Jun 29, 2016

Meanwhile, instead of altering _JAVA_OPTIONS another woraround maybe in installing openjdk-7/8/9-jre-zero - zero jvm is also interpreting-only VM.

Meanwhile, instead of altering _JAVA_OPTIONS another woraround maybe in installing openjdk-7/8/9-jre-zero - zero jvm is also interpreting-only VM.

@leventov

This comment has been minimized.

Show comment
Hide comment
@leventov

leventov Aug 3, 2016

@leninalive -Xint workaround doesn't work for me: java -Xint -version hangs as well at futex(.. according to strace. And even if was working, it would be a suitable solution only for some cases, because Java performance in the interpreter mode is horrible

leventov commented Aug 3, 2016

@leninalive -Xint workaround doesn't work for me: java -Xint -version hangs as well at futex(.. according to strace. And even if was working, it would be a suitable solution only for some cases, because Java performance in the interpreter mode is horrible

@HaloFour

This comment has been minimized.

Show comment
Hide comment
@HaloFour

HaloFour Aug 3, 2016

Agreed. Performance is absolutely horrid and it seems to still hang once it hits a certain point. It took 11 minutes just to compile my Java project via maven. The unit tests started but hung only a couple of tests in. On my MacBook Pro the entire project plus 1100 unit tests plus 380 integration tests takes less than 4 minutes to complete.

HaloFour commented Aug 3, 2016

Agreed. Performance is absolutely horrid and it seems to still hang once it hits a certain point. It took 11 minutes just to compile my Java project via maven. The unit tests started but hung only a couple of tests in. On my MacBook Pro the entire project plus 1100 unit tests plus 380 integration tests takes less than 4 minutes to complete.

@KeithBronstrup

This comment has been minimized.

Show comment
Hide comment
@KeithBronstrup

KeithBronstrup Aug 3, 2016

The -Xint workaround is a no-go for me with Oracle's own JRE, the OpenJDK JRE, and the JRE that ships with IntelliJ IDEA 16. Java 7 JREs work fine, it's just Java 8 that hangs for me, I can confirm @leventov's report as far as when and where it hangs, as well.

The -Xint workaround is a no-go for me with Oracle's own JRE, the OpenJDK JRE, and the JRE that ships with IntelliJ IDEA 16. Java 7 JREs work fine, it's just Java 8 that hangs for me, I can confirm @leventov's report as far as when and where it hangs, as well.

@bitcrazed

This comment has been minimized.

Show comment
Hide comment
@bitcrazed

bitcrazed Aug 11, 2016

Collaborator

@SRGOM Please don't file bugs on UserVoice. UV is for future feature asks, not bug management. The Bash/WSL teams continually engage with our awesome community here on GutHub, but only periodically triage UserVoice.

Collaborator

bitcrazed commented Aug 11, 2016

@SRGOM Please don't file bugs on UserVoice. UV is for future feature asks, not bug management. The Bash/WSL teams continually engage with our awesome community here on GutHub, but only periodically triage UserVoice.

@corneil

This comment has been minimized.

Show comment
Hide comment
@corneil

corneil Aug 27, 2016

I can report that both openjdk-7 and Oracle Java 8 works as expected on Build 14905.rs_prerelease.160811-1739
Without -Xint

corneil commented Aug 27, 2016

I can report that both openjdk-7 and Oracle Java 8 works as expected on Build 14905.rs_prerelease.160811-1739
Without -Xint

@corneil

This comment has been minimized.

Show comment
Hide comment
@corneil

corneil Aug 27, 2016

I was mistaken about oracle java 8. Weird SocketException:

Caused by: java.net.SocketException: Invalid argument
        at java.net.PlainDatagramSocketImpl.datagramSocketCreate(Native Method)
        at java.net.AbstractPlainDatagramSocketImpl.create(AbstractPlainDatagramSocketImpl.java:80)
        at java.net.DatagramSocket.createImpl(DatagramSocket.java:337)
        at java.net.DatagramSocket.<init>(DatagramSocket.java:239)
        at java.net.DatagramSocket.<init>(DatagramSocket.java:299)
        at org.gradle.cache.internal.FileLockCommunicator.<init>(FileLockCommunicator.java:38)

corneil commented Aug 27, 2016

I was mistaken about oracle java 8. Weird SocketException:

Caused by: java.net.SocketException: Invalid argument
        at java.net.PlainDatagramSocketImpl.datagramSocketCreate(Native Method)
        at java.net.AbstractPlainDatagramSocketImpl.create(AbstractPlainDatagramSocketImpl.java:80)
        at java.net.DatagramSocket.createImpl(DatagramSocket.java:337)
        at java.net.DatagramSocket.<init>(DatagramSocket.java:239)
        at java.net.DatagramSocket.<init>(DatagramSocket.java:299)
        at org.gradle.cache.internal.FileLockCommunicator.<init>(FileLockCommunicator.java:38)
@corneil

This comment has been minimized.

Show comment
Hide comment
@corneil

corneil Aug 27, 2016

Same error is present in openjdk-8-jdk

corneil commented Aug 27, 2016

Same error is present in openjdk-8-jdk

@KeithBronstrup

This comment has been minimized.

Show comment
Hide comment
@KeithBronstrup

KeithBronstrup Aug 27, 2016

Any word on when we'll see an official release of that? I'm guessing 2 months or longer?

Any word on when we'll see an official release of that? I'm guessing 2 months or longer?

@sunilmut

This comment has been minimized.

Show comment
Hide comment
@sunilmut

sunilmut Aug 29, 2016

Member

@corneil - Thanks for reporting the issue! Can you list out the steps to repro the socket issues?

Member

sunilmut commented Aug 29, 2016

@corneil - Thanks for reporting the issue! Can you list out the steps to repro the socket issues?

@therealkenc

This comment has been minimized.

Show comment
Hide comment
@therealkenc

therealkenc Aug 29, 2016

Collaborator

@corneil's stack starts at org.gradle.cache.internal.FileLockCommunicator(). Source here. getLocalBindingAddress() is no doubt tripping on #468 and friends.

Collaborator

therealkenc commented Aug 29, 2016

@corneil's stack starts at org.gradle.cache.internal.FileLockCommunicator(). Source here. getLocalBindingAddress() is no doubt tripping on #468 and friends.

@niiiksh

This comment has been minimized.

Show comment
Hide comment
@niiiksh

niiiksh Sep 17, 2016

I have the same problem. Please bring a fix to a release 14393 build, I don't want to install buggy fast ring builds on my Surface Pro 4 :(

niiiksh commented Sep 17, 2016

I have the same problem. Please bring a fix to a release 14393 build, I don't want to install buggy fast ring builds on my Surface Pro 4 :(

@whitecoop whitecoop referenced this issue in technomancy/leiningen Sep 19, 2016

Open

`lein repl` hangs in Bash on Ubuntu on Windows #2185

@jackchammons jackchammons added the java label Oct 24, 2016

@anandbaskaran

This comment has been minimized.

Show comment
Hide comment
@anandbaskaran

anandbaskaran Jan 11, 2017

Any update on SocketException error? I have the same problem

Any update on SocketException error? I have the same problem

@sunilmut

This comment has been minimized.

Show comment
Hide comment
@sunilmut

sunilmut Jan 11, 2017

Member

@anandbaskaran - Which build are you seeing this problem on?

@ALL - If this issue is still open on 15002, please share the repro steps.

Member

sunilmut commented Jan 11, 2017

@anandbaskaran - Which build are you seeing this problem on?

@ALL - If this issue is still open on 15002, please share the repro steps.

@gitbm

This comment has been minimized.

Show comment
Hide comment
@gitbm

gitbm Jun 22, 2017

I had the Java 8 hang problem (I installed the Java 8 JDK from the oracle website and followed some online instructions to install the security certificates and do update-alternatives then ran java -version) on Windows 10 1607/14393.1358

Then I manually triggered a Windows update and the problem went away. The update according to the history was:
2017-06 Update for Windows 10 Version 1607 for x64-based Systems (KB4023834)

So the Windows 10 version is still 1607/14393.1358 but the update seemed to fix it.

In my case I am using a corporate PC which doesn't seem to automatically update from the internet but does an update when manually triggered (In the Update Settings) so most people who get automatic updates probably don't see this problem.

gitbm commented Jun 22, 2017

I had the Java 8 hang problem (I installed the Java 8 JDK from the oracle website and followed some online instructions to install the security certificates and do update-alternatives then ran java -version) on Windows 10 1607/14393.1358

Then I manually triggered a Windows update and the problem went away. The update according to the history was:
2017-06 Update for Windows 10 Version 1607 for x64-based Systems (KB4023834)

So the Windows 10 version is still 1607/14393.1358 but the update seemed to fix it.

In my case I am using a corporate PC which doesn't seem to automatically update from the internet but does an update when manually triggered (In the Update Settings) so most people who get automatic updates probably don't see this problem.

@benhillis benhillis removed the java label May 24, 2018

@therealkenc

This comment has been minimized.

Show comment
Hide comment
@therealkenc

therealkenc May 24, 2018

Collaborator

This issue never followed CONTRIBUTING.md and doesn't have an actionable.

Collaborator

therealkenc commented May 24, 2018

This issue never followed CONTRIBUTING.md and doesn't have an actionable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment