New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net: add mechanism to wait for readability on a TCPConn #15735

Open
bradfitz opened this Issue May 18, 2016 · 24 comments

Comments

Projects
None yet
@bradfitz
Member

bradfitz commented May 18, 2016

EDIT: this proposal has shifted. See #15735 (comment) below.

Old:

The net/http package needs a way to wait for readability on a TCPConn without actually reading from it. (See #15224)

http://golang.org/cl/22031 added such a mechanism, making Read(0 bytes) do a wait for readability, followed by returning (0, nil). But maybe that is strange. Windows already works like that, though. (See new tests in that CL)

Reconsider this for Go 1.8.

Maybe we could add a new method to TCPConn instead, like WaitRead.

@bradfitz bradfitz added this to the Go1.8 milestone May 18, 2016

@bradfitz bradfitz self-assigned this May 18, 2016

@bradfitz

This comment has been minimized.

Show comment
Hide comment
@bradfitz
Member

bradfitz commented May 18, 2016

@gopherbot

This comment has been minimized.

Show comment
Hide comment
@gopherbot

gopherbot commented May 18, 2016

CL https://golang.org/cl/23227 mentions this issue.

gopherbot pushed a commit that referenced this issue May 19, 2016

net: don't return io.EOF from zero byte reads
Updates #15735

Change-Id: I42ab2345443bbaeaf935d683460fc2c941b7679c
Reviewed-on: https://go-review.googlesource.com/23227
Reviewed-by: Ian Lance Taylor <iant@golang.org>

gopherbot pushed a commit that referenced this issue May 19, 2016

net: don't return io.EOF from zero byte reads on Plan 9
Updates #15735.
Fixes #15741.

Change-Id: Ic4ad7e948e8c3ab5feffef89d7a37417f82722a1
Reviewed-on: https://go-review.googlesource.com/23199
Run-TryBot: Mikio Hara <mikioh.mikioh@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
@RalphCorderoy

This comment has been minimized.

Show comment
Hide comment
@RalphCorderoy

RalphCorderoy May 20, 2016

read(2) with a count of zero may be used to detect errors. Linux man page confirms, as does POSIX's read(3p) here. Mentioning it in case it influences this subverting of a Read(0 bytes) not calling syscall.Read.

RalphCorderoy commented May 20, 2016

read(2) with a count of zero may be used to detect errors. Linux man page confirms, as does POSIX's read(3p) here. Mentioning it in case it influences this subverting of a Read(0 bytes) not calling syscall.Read.

@bradfitz

This comment has been minimized.

Show comment
Hide comment
@bradfitz

bradfitz Oct 21, 2016

Member

I found a way to do without this in net/http, so punting to Go 1.9.

Member

bradfitz commented Oct 21, 2016

I found a way to do without this in net/http, so punting to Go 1.9.

@bradfitz bradfitz modified the milestones: Go1.9, Go1.8 Oct 21, 2016

@bradfitz

This comment has been minimized.

Show comment
Hide comment
@bradfitz

bradfitz Dec 12, 2016

Member

Actually, the more I think about this, I don't even want my idle HTTP/RPC goroutines to stick around blocked in a read call. In addition to the array memory backed by the slice given to Read, the goroutine itself is ~4KB of wasted memory.

What I'd really like is a way to register a func() to run when my *net.TCPConn is readable (when a Read call wouldn't block). By analogy, I want the time.AfterFunc efficiency of running a func in a goroutine later, rather than running a goroutine just to block in a time.Sleep.

My new proposal is more like:

package net

// OnReadable runs f in a new goroutine when c is readable;
// that is, when a call to c.Read will not block.
func (c *TCPConn) OnReadable(f func()) {
   // ...
}

Yes, maybe this is getting dangerously into event-based programming land.

Or maybe just the name ("OnWhatever") is offensive. Maybe there's something better.

I would use this in http, http2, and grpc.

/cc @ianlancetaylor @rsc

Member

bradfitz commented Dec 12, 2016

Actually, the more I think about this, I don't even want my idle HTTP/RPC goroutines to stick around blocked in a read call. In addition to the array memory backed by the slice given to Read, the goroutine itself is ~4KB of wasted memory.

What I'd really like is a way to register a func() to run when my *net.TCPConn is readable (when a Read call wouldn't block). By analogy, I want the time.AfterFunc efficiency of running a func in a goroutine later, rather than running a goroutine just to block in a time.Sleep.

My new proposal is more like:

package net

// OnReadable runs f in a new goroutine when c is readable;
// that is, when a call to c.Read will not block.
func (c *TCPConn) OnReadable(f func()) {
   // ...
}

Yes, maybe this is getting dangerously into event-based programming land.

Or maybe just the name ("OnWhatever") is offensive. Maybe there's something better.

I would use this in http, http2, and grpc.

/cc @ianlancetaylor @rsc

@ianlancetaylor

This comment has been minimized.

Show comment
Hide comment
@ianlancetaylor

ianlancetaylor Dec 12, 2016

Contributor

Sounds like you are getting close to #15021.

I'm worried that the existence of such a method will encourage people to start writing their code as callbacks rather than as straightforward goroutines.

Contributor

ianlancetaylor commented Dec 12, 2016

Sounds like you are getting close to #15021.

I'm worried that the existence of such a method will encourage people to start writing their code as callbacks rather than as straightforward goroutines.

@bradfitz

This comment has been minimized.

Show comment
Hide comment
@bradfitz

bradfitz Dec 12, 2016

Member

Yeah. I'm conflicted. I see the benefits and the opportunity for overuse.

Member

bradfitz commented Dec 12, 2016

Yeah. I'm conflicted. I see the benefits and the opportunity for overuse.

@dvyukov

This comment has been minimized.

Show comment
Hide comment
@dvyukov

dvyukov Jan 6, 2017

Member

If we do OnReadable(f func()), won't we need to fork half of standard library for async style? Compress, io, tls, etc readers all assume blocking style and require a blocked goroutine.
I don't see any way to push something asynchronously into e.g. gzip.Reader. Does this mean that I have to choose between no blocked goroutine + my own gzip impl and blocked goroutine + std lib?

Member

dvyukov commented Jan 6, 2017

If we do OnReadable(f func()), won't we need to fork half of standard library for async style? Compress, io, tls, etc readers all assume blocking style and require a blocked goroutine.
I don't see any way to push something asynchronously into e.g. gzip.Reader. Does this mean that I have to choose between no blocked goroutine + my own gzip impl and blocked goroutine + std lib?

@dvyukov

This comment has been minimized.

Show comment
Hide comment
@dvyukov

dvyukov Jan 6, 2017

Member

Re 0-sized reads.
It should work with level-triggered notifications, but netpoll uses epoll in edge-triggered mode (and kqueue iirc). I am concerned if cl/22031 works in more complex cases: waiting for already ready IO, double wait, wait without completely draining read buffer first, etc?

Member

dvyukov commented Jan 6, 2017

Re 0-sized reads.
It should work with level-triggered notifications, but netpoll uses epoll in edge-triggered mode (and kqueue iirc). I am concerned if cl/22031 works in more complex cases: waiting for already ready IO, double wait, wait without completely draining read buffer first, etc?

@bradfitz

This comment has been minimized.

Show comment
Hide comment
@bradfitz

bradfitz Jan 6, 2017

Member

@dvyukov, no, we would only use OnReadable in very high-level places, like the http1 and http2 servers where we know the conn is expected to be idle for long periods of time. The rest of the code underneath would remain in the blocking style.

Member

bradfitz commented Jan 6, 2017

@dvyukov, no, we would only use OnReadable in very high-level places, like the http1 and http2 servers where we know the conn is expected to be idle for long periods of time. The rest of the code underneath would remain in the blocking style.

@dvyukov

This comment has been minimized.

Show comment
Hide comment
@dvyukov

dvyukov Jan 6, 2017

Member

This looks like a half-measure. An http connection can halt in the middle of request...

Member

dvyukov commented Jan 6, 2017

This looks like a half-measure. An http connection can halt in the middle of request...

@bradfitz

This comment has been minimized.

Show comment
Hide comment
@bradfitz

bradfitz Jan 6, 2017

Member

@dvyukov, but not commonly. This would be an optimization for the common case.

Member

bradfitz commented Jan 6, 2017

@dvyukov, but not commonly. This would be an optimization for the common case.

@dvyukov

This comment has been minimized.

Show comment
Hide comment
@dvyukov

dvyukov Jan 7, 2017

Member

An alternative interface can be to register a channel that will receive readiness notifications. The other camp wants this for packet-processing servers, and there starting a goroutine for every packet will be too expensive. However, if at the end you want a goroutine, then the channel will introduce unnecessary overhead.
Channel has a problem with overflow handling (netpoll can't block on send, on the other hand it is not OK to lose notifications).
For completeness, this API should also handle writes.

Member

dvyukov commented Jan 7, 2017

An alternative interface can be to register a channel that will receive readiness notifications. The other camp wants this for packet-processing servers, and there starting a goroutine for every packet will be too expensive. However, if at the end you want a goroutine, then the channel will introduce unnecessary overhead.
Channel has a problem with overflow handling (netpoll can't block on send, on the other hand it is not OK to lose notifications).
For completeness, this API should also handle writes.

@DemiMarie

This comment has been minimized.

Show comment
Hide comment
@DemiMarie

DemiMarie Jan 10, 2017

We need to make sure that this works with Windows IOCP as well.

DemiMarie commented Jan 10, 2017

We need to make sure that this works with Windows IOCP as well.

@rsc

This comment has been minimized.

Show comment
Hide comment
@rsc

rsc Jan 10, 2017

Contributor

Not obvious to me why the API has to handle writes. The thing about reads is that until the data is ready for reading, you can use the memory for other work. If you're waiting to write data, that memory is not reusable (otherwise you'd lose the data you are waiting to write).

Contributor

rsc commented Jan 10, 2017

Not obvious to me why the API has to handle writes. The thing about reads is that until the data is ready for reading, you can use the memory for other work. If you're waiting to write data, that memory is not reusable (otherwise you'd lose the data you are waiting to write).

@dvyukov

This comment has been minimized.

Show comment
Hide comment
@dvyukov

dvyukov Jan 11, 2017

Member

@rsc If we do just 0-sized reads, then write support is not necessary. However, if we do Brad's "My new proposal is more like": func (c *TCPConn) OnReadable(f func()), then this equally applies to writes as well -- to avoid 2 blocked goroutines per connection.

Member

dvyukov commented Jan 11, 2017

@rsc If we do just 0-sized reads, then write support is not necessary. However, if we do Brad's "My new proposal is more like": func (c *TCPConn) OnReadable(f func()), then this equally applies to writes as well -- to avoid 2 blocked goroutines per connection.

@noblehng

This comment has been minimized.

Show comment
Hide comment
@noblehng

noblehng Feb 21, 2017

If memory usage is the concern, it is possible to make long parked G use less memory instead of changing programming style? One main selling point of Go to me is high efficiency network servers without resorting to callbacks.

Something like shrink the stack or move the stack to heap by the GC using some heuristics, that will be littile different from spinning up a new goroutine on callback memory usage wise, and scheduling wise a callback is not much different than goready(). Also I assume the liveness change in Go1.8 could help here too.

For the backing array, if it is preallocated buffer, than a callback doesn't make much different than Read(), maybe it will make some different if it is allocated per-callback and use a pool.

Edit:
Actually we could have some GC deadline or gopark time in runtime.pollDesc, so we could get a list of long parked G from the poller, then GC can kick in, but more dance is still needed to avoid race and make it fast.

noblehng commented Feb 21, 2017

If memory usage is the concern, it is possible to make long parked G use less memory instead of changing programming style? One main selling point of Go to me is high efficiency network servers without resorting to callbacks.

Something like shrink the stack or move the stack to heap by the GC using some heuristics, that will be littile different from spinning up a new goroutine on callback memory usage wise, and scheduling wise a callback is not much different than goready(). Also I assume the liveness change in Go1.8 could help here too.

For the backing array, if it is preallocated buffer, than a callback doesn't make much different than Read(), maybe it will make some different if it is allocated per-callback and use a pool.

Edit:
Actually we could have some GC deadline or gopark time in runtime.pollDesc, so we could get a list of long parked G from the poller, then GC can kick in, but more dance is still needed to avoid race and make it fast.

@noblehng

This comment has been minimized.

Show comment
Hide comment
@noblehng

noblehng Feb 22, 2017

How about a epoll like interface for net.Listener:

type PollableListener interface {
   net.Listener
   // Poll will block till at least one connection been ready for read or write
   // reads and writes are special net.Conn that will not block on EAGAIN
   Poll() (reads []net.Conn, writes []net.Conn)
}

Then the caller of Poll() can has a small number of goroutines to poll for readiness and handle the reads and writes. This should also works well for packet-processing servers.

Note that this only needs to be implemented in the runtime for those Listeners that multiplexed in the kernel, like the net.TCPListener. For other protocol that multiplex in the userspace and doesn't attached to the runtime poller directly, like udp listener or multiplexing streams in a tcp connection, can be implemented outside the runtime. For example, for multiplexing in a tcp connection, we can implemented the epoll like behavior by read from/write to some buffers then poll from them or register callbacks on buffer size changed.

Edit:
To implement this, we can let users of the runtime poller, like socket and os.File, provide a callback function pointer when open the poller for a fd, to notify them the readiness of I/O. The callback should
looks like:

type IOReadyNotify func(mode int32)

And we store this in the runtime.pollDesc, then the runtime.netpollready() function should also call this callback if not nil besides give out the pending goroutine(s).

noblehng commented Feb 22, 2017

How about a epoll like interface for net.Listener:

type PollableListener interface {
   net.Listener
   // Poll will block till at least one connection been ready for read or write
   // reads and writes are special net.Conn that will not block on EAGAIN
   Poll() (reads []net.Conn, writes []net.Conn)
}

Then the caller of Poll() can has a small number of goroutines to poll for readiness and handle the reads and writes. This should also works well for packet-processing servers.

Note that this only needs to be implemented in the runtime for those Listeners that multiplexed in the kernel, like the net.TCPListener. For other protocol that multiplex in the userspace and doesn't attached to the runtime poller directly, like udp listener or multiplexing streams in a tcp connection, can be implemented outside the runtime. For example, for multiplexing in a tcp connection, we can implemented the epoll like behavior by read from/write to some buffers then poll from them or register callbacks on buffer size changed.

Edit:
To implement this, we can let users of the runtime poller, like socket and os.File, provide a callback function pointer when open the poller for a fd, to notify them the readiness of I/O. The callback should
looks like:

type IOReadyNotify func(mode int32)

And we store this in the runtime.pollDesc, then the runtime.netpollready() function should also call this callback if not nil besides give out the pending goroutine(s).

@aajtodd

This comment has been minimized.

Show comment
Hide comment
@aajtodd

aajtodd Feb 27, 2017

I'm fairly new to Go but seeing the callback interface is a little grating given the blocking API exposed everywhere else. Why not expose a public API to the netpoll interfaces?

Go provides no standard public facing event loop (correct me if I'm wrong please). I have need to wait for readability on external FFI socket(s) (given through cgo). It would be nice to re-use the existing netpoll abstraction to also spawn FFI sockets onto rather than having to wrap epoll/IOCP/select. Also I'm guessing wrapping (e.g) epoll from the sys package does not integrate with the scheduler which would also be a bummer.

aajtodd commented Feb 27, 2017

I'm fairly new to Go but seeing the callback interface is a little grating given the blocking API exposed everywhere else. Why not expose a public API to the netpoll interfaces?

Go provides no standard public facing event loop (correct me if I'm wrong please). I have need to wait for readability on external FFI socket(s) (given through cgo). It would be nice to re-use the existing netpoll abstraction to also spawn FFI sockets onto rather than having to wrap epoll/IOCP/select. Also I'm guessing wrapping (e.g) epoll from the sys package does not integrate with the scheduler which would also be a bummer.

@mjgarton

This comment has been minimized.

Show comment
Hide comment
@mjgarton

mjgarton Mar 15, 2017

For a number of my use cases, something like this :

package net

// Readable returns a channel which can be read from whenever a call to c.Read
// would not block.
func (c *TCPConn) Readable() <-chan struct{} {
        // ...
}

.. would be nice because I can select on it. I have no idea whether it's practical to implement this though.

Another alternative (for some of my cases at least) might be somehow enabling reads to be canceled by using a context.

mjgarton commented Mar 15, 2017

For a number of my use cases, something like this :

package net

// Readable returns a channel which can be read from whenever a call to c.Read
// would not block.
func (c *TCPConn) Readable() <-chan struct{} {
        // ...
}

.. would be nice because I can select on it. I have no idea whether it's practical to implement this though.

Another alternative (for some of my cases at least) might be somehow enabling reads to be canceled by using a context.

@funny-falcon

This comment has been minimized.

Show comment
Hide comment
@funny-falcon

funny-falcon Apr 19, 2017

Contributor
type ReadableSig struct {
    C *TCPConn,
    I interface{}
}
// Readable marks connection as waiting for read.
// It will send connection to channel when it becomes readable (or closed by peer)
//  together with additional information passed as 'i' argument.
// Note: if channel blocks, it will create goroutine.
func (c *TCPConn) Readable(i interface{}, ch chan<- ReadableSig)
Contributor

funny-falcon commented Apr 19, 2017

type ReadableSig struct {
    C *TCPConn,
    I interface{}
}
// Readable marks connection as waiting for read.
// It will send connection to channel when it becomes readable (or closed by peer)
//  together with additional information passed as 'i' argument.
// Note: if channel blocks, it will create goroutine.
func (c *TCPConn) Readable(i interface{}, ch chan<- ReadableSig)
@funny-falcon

This comment has been minimized.

Show comment
Hide comment
@funny-falcon

funny-falcon Apr 19, 2017

Contributor

Or even better:

type ReadableNotify struct {
    Conn *TCPConn,
    Ctx context.Context
}
// Readable marks connection as waiting for read and It will send connection
// to channel when it becomes readable (or closed) together with context.
// Internally it may spawn goroutine. It will certainly spawn goroutine if channel
// blocks.
// Note: connections always waits for readability, but if context is closed then
// notification may be omitted. If notification recieved, then Context should be
// checked manually.
func (c *TCPConn) Readable(ctx context.Context, ch chan<- ReadableNotify)

btw, can runtime spawn utility goroutine with less stack than user-faced goroutine?

Contributor

funny-falcon commented Apr 19, 2017

Or even better:

type ReadableNotify struct {
    Conn *TCPConn,
    Ctx context.Context
}
// Readable marks connection as waiting for read and It will send connection
// to channel when it becomes readable (or closed) together with context.
// Internally it may spawn goroutine. It will certainly spawn goroutine if channel
// blocks.
// Note: connections always waits for readability, but if context is closed then
// notification may be omitted. If notification recieved, then Context should be
// checked manually.
func (c *TCPConn) Readable(ctx context.Context, ch chan<- ReadableNotify)

btw, can runtime spawn utility goroutine with less stack than user-faced goroutine?

@gobwas

This comment has been minimized.

Show comment
Hide comment
@gobwas

gobwas Apr 19, 2017

Hi! Great idea 🎆

In our go server with >2millions of alive WebSocket connections we were used epoll to bring the same functionality.

epoll.Handle(conn, EPOLLIN | EPOLLONESHOT, func(events uint) {
    // No more calls will be made for conn until we call epoll.Resume().
    ...
    epoll.Resume(conn)
})

That is, it could be useful to make callback call only once after registration.
In our case we've used EPOLLONESHOT to make this work.

gobwas commented Apr 19, 2017

Hi! Great idea 🎆

In our go server with >2millions of alive WebSocket connections we were used epoll to bring the same functionality.

epoll.Handle(conn, EPOLLIN | EPOLLONESHOT, func(events uint) {
    // No more calls will be made for conn until we call epoll.Resume().
    ...
    epoll.Resume(conn)
})

That is, it could be useful to make callback call only once after registration.
In our case we've used EPOLLONESHOT to make this work.

@amits1995

This comment has been minimized.

Show comment
Hide comment
@amits1995

amits1995 Sep 30, 2017

I think you'll have to add support for Registered IO in Windows

amits1995 commented Sep 30, 2017

I think you'll have to add support for Registered IO in Windows

@bradfitz bradfitz removed their assignment Nov 14, 2017

@bradfitz bradfitz modified the milestones: Go1.10, Unplanned Nov 14, 2017

@bradfitz bradfitz added the Thinking label Nov 14, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment