zpipe reads block on closed writer

[rpedde]

Right now, zpipes implements an infinite pipe. But in order to implement true named pipe semantics over zpipes, it is necessary that the client can detect remote writer close and return a zero byte read.

[hintjens]

I'm wondering how we'd do this with multiple writers.

On Thu, Mar 13, 2014 at 11:58 PM, Ron Pedde notifications@github.comwrote:

Right now, zpipes implements an infinite pipe. But in order to implement
true named pipe semantics over zpipes, it is necessary that the client can
detect remote writer close and return a zero byte read.

Reply to this email directly or view it on GitHubhttps://github.com//issues/4
.

[hintjens]

My thought is, a normal pipe has one writer and N readers; each will get
the "end of pipe" message in time. Separately, we could make an N-to-N pipe
that has its own semantics, e.g. each writer sends an identifiable stream
with its own start and end.

On Fri, Mar 14, 2014 at 12:04 AM, Pieter Hintjens ph@imatix.com wrote:

I'm wondering how we'd do this with multiple writers.

On Thu, Mar 13, 2014 at 11:58 PM, Ron Pedde notifications@github.comwrote:

Right now, zpipes implements an infinite pipe. But in order to implement
true named pipe semantics over zpipes, it is necessary that the client can
detect remote writer close and return a zero byte read.

Reply to this email directly or view it on GitHubhttps://github.com//issues/4
.

[rpedde]

I do think there is advantage to sending start and ends. Right now, we are doing synchronous pipe writes, which I remember you saying limited our throughput significantly. If we send an end state, we could run without acks until the close, and just ack the close and return errors on the close. We'd still have guaranteed delivery (although the error would come at close rather than write), but our throughput would improve. Plus we'd have proper close semantics, at least on a 1-to-1 pipe.

For n-to-n, i think the right implementation is one that tracks real pipe semantics -- open with mode, and track number of reader/writers.

[hintjens]

Asynchronous pipe writes will certainly be faster.

Are there specs somewhere for real n-to-n pipe semantics? I can see some problems trying to keep consistent state across multiple nodes with N writers.

[rpedde]

I don't think the posix definitions are available. Off the top: readers block until a writer connect, writes less than PIPE_BUF (4k on linux, I think) are guaranteed to be atomic, and not intermingled with other writes. Writers block until a reader connects. When the last writer closes, read() returns 0. When the last reader closes, writes against the pipe sigpipe.

I'll try and find more formal specs.

[hintjens]

ZeroVM needs a pub-sub model where data is copied to every reader. Let's
leave n-to-n for a separate thread...

On Fri, Mar 14, 2014 at 10:59 PM, Ron Pedde notifications@github.comwrote:

I don't think the posix definitions are available. Off the top: readers
block until a writer connect, writes less than PIPE_BUF (4k on linux, I
think) are guaranteed to be atomic, and not intermingled with other writes.
Writers block until a reader connects. When the last writer closes, read()
returns 0. When the last reader closes, writes against the pipe sigpipe.

I'll try and find more formal specs.

Reply to this email directly or view it on GitHubhttps://github.com//issues/4#issuecomment-37701921
.

[hintjens]

Fixed by #9