"java.io.IOException: Broken pipe" after foo!< and a key stroke

[eed3si9n]

steps

1. From sbt shell:

   > set TaskKey[Int]("rmtest") := { "rm -i test.txt"!< }
   > rmtest
   rm: test.txt: No such file or directory
   [success] Total time: 0 s, completed Jan 8, 2012 12:39:52 PM
   

2. Then hit enter.

problem

> Exception in thread "Thread-66" java.io.IOException: Broken pipe
    at java.io.FileOutputStream.writeBytes(Native Method)
    at java.io.FileOutputStream.write(FileOutputStream.java:282)
    at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
    at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
    at sbt.BasicIO$.read$1(ProcessImpl.scala:108)
    at sbt.BasicIO$.transferFullyImpl(ProcessImpl.scala:112)
    at sbt.BasicIO$.transferFully(ProcessImpl.scala:88)
    at sbt.BasicIO$.connectToIn(ProcessImpl.scala:79)
    at sbt.BasicIO$$anonfun$input$1.apply(ProcessImpl.scala:80)
    at sbt.BasicIO$$anonfun$input$1.apply(ProcessImpl.scala:80)
    at sbt.SimpleProcessBuilder$$anonfun$2.apply$mcV$sp(ProcessImpl.scala:383)
    at sbt.Spawn$$anon$3.run(ProcessImpl.scala:20)

expectation

No error. (It seems like the character is sent to the already-closed process)

note

This is a minimal repro of a bug reported to sbt-appengine, which spawns appcfg.sh.

[vigdorchik]

This looks fixable if System.in is set to a stream which is muted until we discover the subprocess is gone, re-getting the last value read and buffered and then setting the original System.in.
I'd be happy to contribute a fix.
BTW, same bug manifests itself in scala.sys.process implementation.

[vigdorchik]

This looks fixable if System.in is set to a stream which is muted until we discover the subprocess is gone, re-getting the last value read and buffered and then setting the original System.in.
I'd be happy to contribute a fix.
BTW, same bug manifests itself in scala.sys.process implementation.

[vigdorchik]

Ok, spent some time trying to fix this. Not much luck:
Currently System.in.read() is used to read from standard input, which is not interruptible, so the input is indeed sent to the closed process. It's possible to avoid the exception, but it's not a fix since one would expect subsequent input after process is done to be handled by sbt itself.
Having read() interrupted when using nio channels also doesn't work since interrupt closes the underlying stream, so System.in is gone and sbt crashes.

[vigdorchik]

Ok, spent some time trying to fix this. Not much luck:
Currently System.in.read() is used to read from standard input, which is not interruptible, so the input is indeed sent to the closed process. It's possible to avoid the exception, but it's not a fix since one would expect subsequent input after process is done to be handled by sbt itself.
Having read() interrupted when using nio channels also doesn't work since interrupt closes the underlying stream, so System.in is gone and sbt crashes.

[harrah]

Is there an actual problem here or can the Broken pipe exception just be swallowed?

[harrah]

Is there an actual problem here or can the Broken pipe exception just be swallowed?

[eed3si9n]

@harrah if I run rmtest and then test, since t is eaten by the zombie, sbt gets est:

[error] Not a valid command: est (similar: set, last, exit)
[error] Not a valid project ID: est
[error] Not a valid configuration: est (similar: test)
[error] Not a valid key: est (similar: test, rmtest, site)
[error] est
[error]    ^

[eed3si9n]

@harrah if I run rmtest and then test, since t is eaten by the zombie, sbt gets est:

[error] Not a valid command: est (similar: set, last, exit)
[error] Not a valid project ID: est
[error] Not a valid configuration: est (similar: test)
[error] Not a valid key: est (similar: test, rmtest, site)
[error] est
[error]    ^

[vigdorchik]

I think the best solution would be to display "Process finished, press any key to continue" message rather than going directly to sbt prompt.

[vigdorchik]

I think the best solution would be to display "Process finished, press any key to continue" message rather than going directly to sbt prompt.

[harrah]

Ah, I see Eugene V. answered my question before I asked it. For the suggestion to prompt to continue, I don't think that solution works in general.

I expect that this is a fundamental problem with the java.lang.Process interface. The specific problem of piping stdin to a process can probably be fixed using the improved redirection supported by Java 7.

[harrah]

Ah, I see Eugene V. answered my question before I asked it. For the suggestion to prompt to continue, I don't think that solution works in general.

I expect that this is a fundamental problem with the java.lang.Process interface. The specific problem of piping stdin to a process can probably be fixed using the improved redirection supported by Java 7.

[vigdorchik]

Java7 indeed solves the problem. Expect a pull request soon.

[vigdorchik]

Java7 indeed solves the problem. Expect a pull request soon.

[inkytonik]

Mark suggests that a problem I am having with input and sbt is the same as this issue, so he suggested I add the details here. Note that I am using Java 7 which is perhaps evidence that this is a different problem, since Eugene notes above that Java 7 fixes this issue.

Setup: Mac OS X 10.7.3, sbt 0.11.2, JDK 1.7.0_04-b21

I am running a program that uses JLine 2.6. To avoid a clash with the version
of JLine that sbt uses, my build.sbt has

fork in run := true
connectInput in run := true
outputStrategy in run := Some (StdoutOutput)

Here's a log of the run:

kiama 1.2.1-SNAPSHOT> test:run-main org.kiama.example.lambda.Lambda
Enter lambda calculus expressions for evaluation.
lambda> (\x . x) 4
4
lambda> 
[success] Total time: 42 s, completed 08/05/2012 8:59:05 AM
kiama 1.2.1-SNAPSHOT> Exception in thread "Thread-1" java.io.IOException: Stream closed
    at java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:434)
    at java.io.OutputStream.write(OutputStream.java:116)
    at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
    at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
    at sbt.BasicIO$.read$1(ProcessImpl.scala:108)
    at sbt.BasicIO$.transferFullyImpl(ProcessImpl.scala:112)
    at sbt.BasicIO$.transferFully(ProcessImpl.scala:88)
    at sbt.BasicIO$.connectToIn(ProcessImpl.scala:79)
    at sbt.BasicIO$$anonfun$input$1.apply(ProcessImpl.scala:80)
    at sbt.BasicIO$$anonfun$input$1.apply(ProcessImpl.scala:80)
    at sbt.SimpleProcessBuilder$$anonfun$2.apply$mcV$sp(ProcessImpl.scala:383)
    at sbt.Spawn$$anon$3.run(ProcessImpl.scala:20)

kiama 1.2.1-SNAPSHOT> 
kiama 1.2.1-SNAPSHOT> 

So, I run the program using "test:run-main". All is well. JLine operates correctly and the program receives the input that it should receive.

I exit the program using ^D (end of file) which correctly returns me to the sbt prompt. Then if I hit RETURN, I get the exception shown, which indicates that sbt's input stream has been closed, presumably because the ^D from the program run has been seen by it. Another RETURN gets me the prompt again and everything seems to be ok from that point onwards.

[inkytonik]

Mark suggests that a problem I am having with input and sbt is the same as this issue, so he suggested I add the details here. Note that I am using Java 7 which is perhaps evidence that this is a different problem, since Eugene notes above that Java 7 fixes this issue.

Setup: Mac OS X 10.7.3, sbt 0.11.2, JDK 1.7.0_04-b21

I am running a program that uses JLine 2.6. To avoid a clash with the version
of JLine that sbt uses, my build.sbt has

fork in run := true
connectInput in run := true
outputStrategy in run := Some (StdoutOutput)

Here's a log of the run:

kiama 1.2.1-SNAPSHOT> test:run-main org.kiama.example.lambda.Lambda
Enter lambda calculus expressions for evaluation.
lambda> (\x . x) 4
4
lambda> 
[success] Total time: 42 s, completed 08/05/2012 8:59:05 AM
kiama 1.2.1-SNAPSHOT> Exception in thread "Thread-1" java.io.IOException: Stream closed
    at java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:434)
    at java.io.OutputStream.write(OutputStream.java:116)
    at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
    at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
    at sbt.BasicIO$.read$1(ProcessImpl.scala:108)
    at sbt.BasicIO$.transferFullyImpl(ProcessImpl.scala:112)
    at sbt.BasicIO$.transferFully(ProcessImpl.scala:88)
    at sbt.BasicIO$.connectToIn(ProcessImpl.scala:79)
    at sbt.BasicIO$$anonfun$input$1.apply(ProcessImpl.scala:80)
    at sbt.BasicIO$$anonfun$input$1.apply(ProcessImpl.scala:80)
    at sbt.SimpleProcessBuilder$$anonfun$2.apply$mcV$sp(ProcessImpl.scala:383)
    at sbt.Spawn$$anon$3.run(ProcessImpl.scala:20)

kiama 1.2.1-SNAPSHOT> 
kiama 1.2.1-SNAPSHOT> 

So, I run the program using "test:run-main". All is well. JLine operates correctly and the program receives the input that it should receive.

I exit the program using ^D (end of file) which correctly returns me to the sbt prompt. Then if I hit RETURN, I get the exception shown, which indicates that sbt's input stream has been closed, presumably because the ^D from the program run has been seen by it. Another RETURN gets me the prompt again and everything seems to be ok from that point onwards.

[harrah]

Eugene meant that Java 7 allows us to solve the problem, not that it automatically solves the problem.

[harrah]

Eugene meant that Java 7 allows us to solve the problem, not that it automatically solves the problem.

[inkytonik]

Ah, ok. My mistake.

[inkytonik]

Ah, ok. My mistake.

[harrah]

Fixed by @vigdorchik in 53ee1c5..dabb0fd (for Java 7 only).

[harrah]

Fixed by @vigdorchik in 53ee1c5..dabb0fd (for Java 7 only).