New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

os: API to detect that the read end of a pipe was closed #26049

Open
ibukanov opened this Issue Jun 25, 2018 · 9 comments

Comments

Projects
None yet
5 participants
@ibukanov

ibukanov commented Jun 25, 2018

Currently there is no API in Go to detect that the read end of a pipe was closed without writing to the pipe a non-empty slice. This prevents, for example, to write in a safe Go a version of GNU tail utility that exits immediately when it detects that the read end of its stdout was closed even when it waits for more input like when it is called via tail -f. For example, given the following case:

tail -f file-to-follow | grep -q foo

GNU tail exits immediately after the grep finds the word foo and terminates without waiting that the file-to-follow will be extended and trying to write those extra bytes to stdout.

Yet to write such functionality in Go one needs to use unsafe code to call sys.Select or similar and wait for an syscall.EPIPE from the stdout descriptor.

What version of Go are you using (go version)?

go1.10.3

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

amd64 linux fedora-28

@ianlancetaylor ianlancetaylor changed the title from API to detect that the read end of a pipe was closed to os: API to detect that the read end of a pipe was closed Jun 25, 2018

@ianlancetaylor ianlancetaylor added this to the Unplanned milestone Jun 25, 2018

@ianlancetaylor

This comment has been minimized.

Contributor

ianlancetaylor commented Jun 25, 2018

Do you have a proposal for how to address this?

@ibukanov

This comment has been minimized.

ibukanov commented Jun 25, 2018

An API that is sufficient to implement that tail-like functionality is to add to os.signal something like

func ClosedPipe(pipe *os.File) (<-chan struct{}, error)

that returns a channel that becomes ready when the read end of the pipe closes. It is very OK if this only works with pipes where SetWriteDeadline works, i.e. the pipes that the poller supports.

Alternatively the API can take a callback that is called when the pipe is closed, i.e. like

func ClosedPipe(pipe *os.File, callback func()) error

The nice thing of this form is that one can pass there CancelFunc from context.WithCancel() as a typical reaction to the closed read end of the pipe is to stop current activities that in future result in writes to the pipe.

@robpike

This comment has been minimized.

Contributor

robpike commented Jun 25, 2018

How do I detect that the read end has been closed?

@ibukanov

This comment has been minimized.

ibukanov commented Jun 25, 2018

@robpike On Linux when the read end is closed poll/epoll syscalls for the write end of the pipe return POLLERR/EPOLLERR.

@robpike robpike closed this Jun 26, 2018

@cespare

This comment has been minimized.

Contributor

cespare commented Jun 26, 2018

@robpike was this close intentional?

@robpike robpike reopened this Jun 26, 2018

@ibukanov

This comment has been minimized.

ibukanov commented Jun 26, 2018

On systems with kqueue the interface also provides notifications about the closed read end of the pipe. But on Windows it seems with anonymous pipes the only way to support this is via periodic calls to the write function with zero-length buffer. At least the question on StackOverflow has not got a better answer, https://stackoverflow.com/questions/11564166/detecting-if-anonymous-pipe-is-writable-to-detect-when-to-end-process

@FiloSottile FiloSottile modified the milestones: Unplanned, Proposal Jun 26, 2018

@robpike

This comment has been minimized.

Contributor

robpike commented Jun 26, 2018

Almost 50 years on and we still don't understand how to handle a closed pipe. It's remarkable how much energy has been consumed on this topic, how many crashes caused, how many solutions proposed.

I would be more comfortable about this if there was a non-epoll, non-signal-catching way to learn about a closed pipe. Is there?

@ibukanov

This comment has been minimized.

ibukanov commented Jun 26, 2018

@robpike there is no such API besides listening for epoll/poll/kqueue events.

@robpike

This comment has been minimized.

Contributor

robpike commented Jun 27, 2018

@ibukanov I am not surprised. As I said, 50 years on....

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment