IO.to_outputstream (a.k.a. RubyIO.getOutStream) returns a non-writable stream in some cases.

[raelik]

While doing some testing with chunked responses on a plain Rack app running under puma 2.11.3 (using rack.hijack to get access to the raw socket) on jruby 1.7.20.1, I was getting a "not opened for writing" IOException, which I traced back to checkWritable() in the ChannelStream class used internally by RubyIO. The OutputStreamAdapter ultimately returned by getOutStream() uses checkWritable() in many places, so I was able to get around this issue by using getChannel() instead and wrapping it in my own OutputStream using Channels.newOutputStream(). This issue has been resolved in 9k via the overhaul of the IO system.

[headius]

So was it returning an OutputStream wrapping a read-only channel? Channels.toOutputStream should be doing the right thing...

[headius]

Ok, had to refresh my memory a bit.

I see that we create our own OutputStreamAdapter that provides all the OutputStream behavior. This calls back into ChannelStream, which checks whether the stream was opened with write mode. If the IO is not set up for write, why are you trying to get an OutputStream from it? I'm a little confused...some example code might help. Or a patch, if you have an idea how you'd like to solve this!

[raelik]

The specific case I'm using it for is with the socket that puma passes in to a rack.hijack lambda to do chunked encoding. Here's a hello world example of a chunked response using rack.hijack:

def run
  output = "Hello World!"
  hijack = -> socket {
    socket.print(("%x\r\n" % output.bytesize) + output + "\r\n0\r\n\r\n")
  }
  [200, {'Transfer-Encoding' => 'chunked', 'rack.hijack' => hijack, nil]
end

Passing a lambda to the rack.hijack "header" tells rack that you want to do a partial hijack of the connection and gain full control of the client connection (writable). In my particular case, I'm iterating over a very large JDBC ResultSet and am streaming CSV back to the client in chunks. I've implemented this iteration and CSV conversion in Java for performance, and am using the Apache HTTP Commons library to handle the chunked encoding, using ChunkedOutputStream and the SessionOutputBufferImpl classes. So in my case, the lambda above looks more like this:

res = stmt.execute_query(sql)
-> socket { begin
  writer = ResultSetCsvWriter(socket.to_outputstream, CHUNK_SIZE)
  writer.write_all_rows_and_close(res)
ensure
  res.close
  stmt.close
  conn.close
end }

ResultSetCsvWriter is my Java class, with the following constructor:

  public ResultSetCsvWriter(OutputStream stream, int buffer_size) {
    int chunk_size = buffer_size / 4;
    int chunk_buffer = chunk_size + ((int) Math.floor(Math.log(chunk_size) / Math.log(16))) + 5;
    metrics = new HttpTransportMetricsImpl();
    buffer = new SessionOutputBufferImpl(metrics, chunk_buffer);
    buffer.bind(stream);
    chunked = new ChunkedOutputStream(chunk_size, buffer);
    writer = new BufferedWriter(new OutputStreamWriter(chunked), buffer_size);
  }

writeAllRowsAndClose does exactly what you would expect, iterating over the rows in the result set, and each column in those rows, writing it all to the BufferedWriter with interspersed commas and line feeds, closing the writer and ChunkedOutputStream when it's done.

[headius]

My confusion is why you'd be seeing "not opened for writing" if you're using our OutputStreamAdapter against a channel opened for write. Or put differently...if it's opened for write, our adapter should work fine. If it's not opened for write, then how can any output work? What exactly is the problem with JRuby we need to fix?

[raelik]

I'm as similarly confused as you are, as that is exactly what seems to be happening. The channel is definitely opened for write, as I can use getChannel() and wrap it in my own OutputStream to write to it (or just write to it in Ruby). Something is failing with regards to the OutputStreamAdapter. Whether that failure has to do with the OutputStreamAdapter itself, or something that puma is doing to the socket before it calls the lambda which causes it to not be able to write to the socket, I don't know.

[headius]

Can you set up a git repo that reproduces the issue for me?

[headius]

I'm wondering if this might be the same issue as #3189, where a stream was getting closed for both read and write when requesting it be closed just for write. Are you on Windows?

[raelik]

No, this was on Linux. Sorry I haven't set up a repo/gist to reproduce the issue, been caught up in work fires.

[enebo]

1.7.x is EOL and this will not get fixed. Marking WONTFIX