New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IO.popen with file mode w does not close stream and hangs on Win7 #3473

Closed
robertpanzer opened this Issue Nov 20, 2015 · 8 comments

Comments

Projects
None yet
2 participants
@robertpanzer

This issue is related and probably the reason for asciidoctor/asciidoctorj#409

I have the following test ruby script that pipes a string to a graphviz process.
(So to retest graphviz should be installed on the machine.)
I experienced the issue with JRuby 9.0.4.0, with JRuby 1.7.+ everything works fine.

If you start this script it simply hangs:

args = ['C:\Tools\graphviz\bin\dot.exe', '-oimage.png', '-Tpng']
input = <<EOS
digraph G {
    A -> B
}
EOS

IO.popen(args, 'w') { |io|
    io.write input
}

I think the reason is that JRuby thinks that the stream is already closed in https://github.com/jruby/jruby/blob/master/core/src/main/java/org/jruby/RubyIO.java#L3900 and does not close the stream.
Then the following process.waitFor() hangs until the end of days.

@robertpanzer

This comment has been minimized.

Show comment
Hide comment
@robertpanzer

robertpanzer Nov 20, 2015

BTW just tested that this runs successfully on centos:

args = ['/usr/bin/dot', '-oimage.png', '-Tpng']
input = <<EOS
digraph G {
        A -> B
}
EOS

IO.popen(args, 'w') { |io|
        io.write input
}

BTW just tested that this runs successfully on centos:

args = ['/usr/bin/dot', '-oimage.png', '-Tpng']
input = <<EOS
digraph G {
        A -> B
}
EOS

IO.popen(args, 'w') { |io|
        io.write input
}
@headius

This comment has been minimized.

Show comment
Hide comment
@headius

headius Mar 14, 2016

Member

No investigation yet. Windows does use the old logic, since we have not completed native process support under win32 yet. However, that old logic should be at least as functional as 1.7, so I'm not sure why this would have regressed yet.

Ultimate solution is obviously getting full native process support on Windows, but that will take a bit more time.

Member

headius commented Mar 14, 2016

No investigation yet. Windows does use the old logic, since we have not completed native process support under win32 yet. However, that old logic should be at least as functional as 1.7, so I'm not sure why this would have regressed yet.

Ultimate solution is obviously getting full native process support on Windows, but that will take a bit more time.

@headius

This comment has been minimized.

Show comment
Hide comment
@headius

headius Mar 14, 2016

Member

Brief investigation found that I cannot reproduce this issue on OS X by specifying -Xnative.enabled=false. A simple popen with a block runs and returns successfully.

Member

headius commented Mar 14, 2016

Brief investigation found that I cannot reproduce this issue on OS X by specifying -Xnative.enabled=false. A simple popen with a block runs and returns successfully.

@robertpanzer

This comment has been minimized.

Show comment
Hide comment
@robertpanzer

robertpanzer Mar 14, 2016

Wow, it blocks on my OS X!
Didn't know this switch yet.

This is where the main thread hangs:

"main" #1 prio=5 os_prio=31 tid=0x00007ffec1002800 nid=0x1703 in Object.wait() [0x0000700000318000]
   java.lang.Thread.State: WAITING (on object monitor)
    at java.lang.Object.wait(Native Method)
    - waiting on <0x00000007b70614b0> (a java.lang.UNIXProcess)
    at java.lang.Object.wait(Object.java:502)
    at java.lang.UNIXProcess.waitFor(UNIXProcess.java:396)
    - locked <0x00000007b70614b0> (a java.lang.UNIXProcess)
    at org.jruby.util.ShellLauncher$POpenProcess.waitFor(ShellLauncher.java:1021)
    at org.jruby.RubyIO.popen(RubyIO.java:3892)
    at org.jruby.RubyIO$INVOKER$s$0$2$popen.call(RubyIO$INVOKER$s$0$2$popen.gen)
    at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:209)
    at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:343)
    at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:205)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:209)
    at test.invokeOther7:popen(test.rb)
    at test.RUBY$script(test.rb:8)
    at java.lang.invoke.LambdaForm$DMH/868693306.invokeStatic_L7_L(LambdaForm$DMH)
    at java.lang.invoke.LambdaForm$BMH/94345706.reinvoke(LambdaForm$BMH)
    at java.lang.invoke.LambdaForm$MH/684874119.invoker(LambdaForm$MH)
    at java.lang.invoke.LambdaForm$MH/670035812.invokeExact_MT(LambdaForm$MH)
    at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:627)
    at org.jruby.ir.Compiler$1.load(Compiler.java:111)
    at org.jruby.Ruby.runScript(Ruby.java:817)
    at org.jruby.Ruby.runScript(Ruby.java:809)
    at org.jruby.Ruby.runNormally(Ruby.java:747)
    at org.jruby.Ruby.runFromMain(Ruby.java:569)
    at org.jruby.Main.doRunFromMain(Main.java:415)
    at org.jruby.Main.internalRun(Main.java:310)
    at org.jruby.Main.run(Main.java:239)
    at org.jruby.Main.main(Main.java:201)

Did you try with dot?

Maybe there is something particular about graphviz.
For example I think it will not write anything into its output until it received the full input.

Tried with JRuby 9.0.5.0 I just installed via brew.

My test program was:

args = ['/usr/local/bin/dot', '-oimage.png', '-Tpng']
input = <<EOS
digraph G {
        A -> B
}
EOS

IO.popen(args, 'w') { |io|
        io.write input
}

/usr/local/bin/dot is installed via brew install graphviz.

Wow, it blocks on my OS X!
Didn't know this switch yet.

This is where the main thread hangs:

"main" #1 prio=5 os_prio=31 tid=0x00007ffec1002800 nid=0x1703 in Object.wait() [0x0000700000318000]
   java.lang.Thread.State: WAITING (on object monitor)
    at java.lang.Object.wait(Native Method)
    - waiting on <0x00000007b70614b0> (a java.lang.UNIXProcess)
    at java.lang.Object.wait(Object.java:502)
    at java.lang.UNIXProcess.waitFor(UNIXProcess.java:396)
    - locked <0x00000007b70614b0> (a java.lang.UNIXProcess)
    at org.jruby.util.ShellLauncher$POpenProcess.waitFor(ShellLauncher.java:1021)
    at org.jruby.RubyIO.popen(RubyIO.java:3892)
    at org.jruby.RubyIO$INVOKER$s$0$2$popen.call(RubyIO$INVOKER$s$0$2$popen.gen)
    at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:209)
    at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:343)
    at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:205)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:209)
    at test.invokeOther7:popen(test.rb)
    at test.RUBY$script(test.rb:8)
    at java.lang.invoke.LambdaForm$DMH/868693306.invokeStatic_L7_L(LambdaForm$DMH)
    at java.lang.invoke.LambdaForm$BMH/94345706.reinvoke(LambdaForm$BMH)
    at java.lang.invoke.LambdaForm$MH/684874119.invoker(LambdaForm$MH)
    at java.lang.invoke.LambdaForm$MH/670035812.invokeExact_MT(LambdaForm$MH)
    at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:627)
    at org.jruby.ir.Compiler$1.load(Compiler.java:111)
    at org.jruby.Ruby.runScript(Ruby.java:817)
    at org.jruby.Ruby.runScript(Ruby.java:809)
    at org.jruby.Ruby.runNormally(Ruby.java:747)
    at org.jruby.Ruby.runFromMain(Ruby.java:569)
    at org.jruby.Main.doRunFromMain(Main.java:415)
    at org.jruby.Main.internalRun(Main.java:310)
    at org.jruby.Main.run(Main.java:239)
    at org.jruby.Main.main(Main.java:201)

Did you try with dot?

Maybe there is something particular about graphviz.
For example I think it will not write anything into its output until it received the full input.

Tried with JRuby 9.0.5.0 I just installed via brew.

My test program was:

args = ['/usr/local/bin/dot', '-oimage.png', '-Tpng']
input = <<EOS
digraph G {
        A -> B
}
EOS

IO.popen(args, 'w') { |io|
        io.write input
}

/usr/local/bin/dot is installed via brew install graphviz.

@headius

This comment has been minimized.

Show comment
Hide comment
@headius

headius Mar 14, 2016

Member

It does hang for me using dot (I was using echo).

Member

headius commented Mar 14, 2016

It does hang for me using dot (I was using echo).

@robertpanzer

This comment has been minimized.

Show comment
Hide comment
@robertpanzer

robertpanzer Mar 14, 2016

\o/ First goal reached :)

Am Montag, 14. März 2016 schrieb Charles Oliver Nutter :

It does hang for me using dot (I was using echo).


Reply to this email directly or view it on GitHub
#3473 (comment).

\o/ First goal reached :)

Am Montag, 14. März 2016 schrieb Charles Oliver Nutter :

It does hang for me using dot (I was using echo).


Reply to this email directly or view it on GitHub
#3473 (comment).

@headius

This comment has been minimized.

Show comment
Hide comment
@headius

headius Mar 14, 2016

Member

@robertpanzer Your theory appears to be on the right track... io.openFile.isOpen() is returning false, so close never gets called. I think I know why.

Member

headius commented Mar 14, 2016

@robertpanzer Your theory appears to be on the right track... io.openFile.isOpen() is returning false, so close never gets called. I think I know why.

@headius headius closed this in 16aef81 Mar 14, 2016

@headius

This comment has been minimized.

Show comment
Hide comment
@headius

headius Mar 14, 2016

Member

Ok, so it's fixed, but I'm not sure what to do about testing. We are currently working on improving Windows test coverage, which may pick this up. We also are not running any suites with native.enabled=false, and doing that would certainly give us some failures to work through.

Member

headius commented Mar 14, 2016

Ok, so it's fixed, but I'm not sure what to do about testing. We are currently working on improving Windows test coverage, which may pick this up. We also are not running any suites with native.enabled=false, and doing that would certainly give us some failures to work through.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment