net: add mechanism to wait for readability on a TCPConn #15735
Comments
/cc @ianlancetaylor @rsc |
CL https://golang.org/cl/23227 mentions this issue. |
Updates #15735 Change-Id: I42ab2345443bbaeaf935d683460fc2c941b7679c Reviewed-on: https://go-review.googlesource.com/23227 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Updates #15735. Fixes #15741. Change-Id: Ic4ad7e948e8c3ab5feffef89d7a37417f82722a1 Reviewed-on: https://go-review.googlesource.com/23199 Run-TryBot: Mikio Hara <mikioh.mikioh@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
read(2) with a count of zero may be used to detect errors. Linux man page confirms, as does POSIX's read(3p) here. Mentioning it in case it influences this subverting of a Read(0 bytes) not calling syscall.Read. |
I found a way to do without this in net/http, so punting to Go 1.9. |
Actually, the more I think about this, I don't even want my idle HTTP/RPC goroutines to stick around blocked in a read call. In addition to the array memory backed by the slice given to What I'd really like is a way to register a My new proposal is more like: package net
// OnReadable runs f in a new goroutine when c is readable;
// that is, when a call to c.Read will not block.
func (c *TCPConn) OnReadable(f func()) {
// ...
} Yes, maybe this is getting dangerously into event-based programming land. Or maybe just the name (" I would use this in http, http2, and grpc. /cc @ianlancetaylor @rsc |
Sounds like you are getting close to #15021. I'm worried that the existence of such a method will encourage people to start writing their code as callbacks rather than as straightforward goroutines. |
Yeah. I'm conflicted. I see the benefits and the opportunity for overuse. |
If we do OnReadable(f func()), won't we need to fork half of standard library for async style? Compress, io, tls, etc readers all assume blocking style and require a blocked goroutine. |
Re 0-sized reads. |
@dvyukov, no, we would only use |
This looks like a half-measure. An http connection can halt in the middle of request... |
@dvyukov, but not commonly. This would be an optimization for the common case. |
An alternative interface can be to register a channel that will receive readiness notifications. The other camp wants this for packet-processing servers, and there starting a goroutine for every packet will be too expensive. However, if at the end you want a goroutine, then the channel will introduce unnecessary overhead. |
We need to make sure that this works with Windows IOCP as well. |
Not obvious to me why the API has to handle writes. The thing about reads is that until the data is ready for reading, you can use the memory for other work. If you're waiting to write data, that memory is not reusable (otherwise you'd lose the data you are waiting to write). |
@rsc If we do just 0-sized reads, then write support is not necessary. However, if we do Brad's "My new proposal is more like": |
If memory usage is the concern, it is possible to make long parked G use less memory instead of changing programming style? One main selling point of Go to me is high efficiency network servers without resorting to callbacks. Something like shrink the stack or move the stack to heap by the GC using some heuristics, that will be littile different from spinning up a new goroutine on callback memory usage wise, and scheduling wise a callback is not much different than For the backing array, if it is preallocated buffer, than a callback doesn't make much different than Edit: |
How about a epoll like interface for type PollableListener interface {
net.Listener
// Poll will block till at least one connection been ready for read or write
// reads and writes are special net.Conn that will not block on EAGAIN
Poll() (reads []net.Conn, writes []net.Conn)
} Then the caller of Note that this only needs to be implemented in the runtime for those Listeners that multiplexed in the kernel, like the Edit: type IOReadyNotify func(mode int32) And we store this in the |
I'm fairly new to Go but seeing the callback interface is a little grating given the blocking API exposed everywhere else. Why not expose a public API to the netpoll interfaces? Go provides no standard public facing event loop (correct me if I'm wrong please). I have need to wait for readability on external FFI socket(s) (given through cgo). It would be nice to re-use the existing netpoll abstraction to also spawn FFI sockets onto rather than having to wrap epoll/IOCP/select. Also I'm guessing wrapping (e.g) epoll from the sys package does not integrate with the scheduler which would also be a bummer. |
For a number of my use cases, something like this :
.. would be nice because I can select on it. I have no idea whether it's practical to implement this though. Another alternative (for some of my cases at least) might be somehow enabling reads to be canceled by using a context. |
typical Go style is quite convenient for high-level nd medium-level logic, and quite inefficient at low level. |
@ianlancetaylor as exploration, maybe option of exporting network poller API would be better idea than a callback? I mean something like this: import "net/poll"
p, err := poll.Create(...) // epoll_create() on Linux.
if err != nil {
// handle error
}
go func() {
for ev := range poll.Wait() { // epoll_wait on Linux.
ev.Conn.Read(...)
ev.Data // User data bound within Notify() call.
}
}()
conn, err := net.Dial(...)
if err != nil {
// handle error
}
if err := p.Notify(poll.Read, conn, data); err != nil { // epoll_ctl on Linux.
// handle error
} Thus, we may leave general poller untouched (and non-blocking), but provide additional polling mechanism which will block when poll.Wait() stopped being drained. Also Notify() api may differ, e.g. accepting different channels to write event to, as you suggested, but in blocking manner – without events loss and with strong notice in method comment =) |
@dvyukov, would it work to do a non-blocking read of a byte or two internally to verify readability before raising a notification? That adds at least one syscall to the overhead, but for a network or storage situation, maybe that's OK. In the spirit of epoll, it would be helpful to get notifications for an arbitrary set of io.Reader objects via a single channel, since we cannot |
This looks dirty and slow and in the end there can be just 1 byte. So it was readable, but not now anymore ;)
I think this is supported by the Ian's proposal, no? |
Then let r.Read() be non-blocking after r.NotifyWhenReadable() and possibly return error io.WouldBlock? |
@networkimprov JFYI there is an ability to Select() on runtime defined set of channels via reflect api. |
@networkimprov Having a non-blocking @gobwas I think that in practice using a poller is going to lead people to write code using callback APIs. It's the most natural way to use a poller. |
Do you have a plan to deal with the issue Dmitry raised? |
@networkimprov Which issue? If you mean the problem with possibly losing a channel notification, then of course he is quite right, and the channel send must be a blocking send. Depending on the implementation, this may require spinning up a goroutine to do the send. |
No, I was referring to
How do you deal with this without letting r.Read() be non-blocking after r.NotifyWhenReadable() and possibly returning error io.WouldBlock? |
The problem I was trying to address was "large server with a lot of goroutines burns a lot of memory in buffers waiting to read." For that purpose it's OK if a spurious notification leads to a blocking |
Brad was trying to get rid of the idle goroutines: #15735 (comment) If that's not a requirement, an alternate read API could internally poll, malloc/free, and do non-blocking reads, and return a new buffer on success. An internal buffer pool would avoid thrashing the allocator due to fruitless reads after spurious poll events.
|
My channel suggestion also permits getting rid of the idle goroutines, for people who want to write a more complex program. |
But it does incur a read-buffer + idle goroutine for every spurious poll event, which could be costly. @dvyukov, any idea as to the proportion of spurious epoll wakeups in Linux? More on epoll difficulties (from 2y ago): |
No, I don't. |
I am trying to understand when a tcp server has closed the connection, as I am not receiving errors on the reads. Is there any reliable way to detect that a TCP connection was closed from the server without relying on the server writing something on the connection ? |
FYI, we implemented this in mysql driver. Note that it works only on Unix, and it caused several allocations. |
It's highly useful to reduce overhead when the go program holds a lot of connections (connection pool) and now I'm using the Looking forward to land in Go, many thanks:) |
@ianlancetaylor + @bradfitz a typical problem I have in an http proxy is, that connection spikes can create spikes in memory usage. I think this can be fixed with using epoll and I hope your approach will cover the problem. We would need to be able to set the max concurrency level for the goroutine calls, that will read and write from/to sockets. |
I need readability notification badly in my project, as I have to pre-allocate 4k buffer per connection before conn.Read([]byte), just like io.Copy does: https://golang.org/src/io/io.go?s=12796:12856#L399 UPDATE: |
So there is RawConn now which has an interface very much like what @bradfitz was proposing here. However, it calls the read callback before calling wait for read. It must do this, as the net poller uses edge-triggered events - they won't fire if there is already data on the socket. One workaround is to use a small stack buffer for the initial Read, and then when that reads some data, allocate and copy it to a real buffer, and then call Read again. That helps, but you'll still have the goroutine's 4KB stack overhead. Another option is use RawConn.Control with I'd still like to see an approach (maybe an async OnReadable callback method on RawConn, like bradfitz is proposing here?) to avoid having the goroutine overhead. Mail.ru avoids the net poller entirely and manually manages epoll for precisely this reason, as 4KB per WebSocket connection with millions of open but mostly idle WebSockets waiting for new mail is just too much overhead. |
I don't think OnReadable callback is a good idea. If I have 2 handlers to toggle based on incoming data, then it's not impossible to code this kind of nested callbacks which reference to each other. For such reason, in order to writing comprehensive logic we have to copy the buffer from callback out to another goroutine, in such case, the memory usage will be out of control, as we lost the ability to back-pressure the congestion signal to senders. (Before, we won't start Even for a callback like In all these cases above, we still have inactive 4KB-buffers somewhere. |
I wrote a library based on ideas above, golang async-io |
For my particular use case having a non-blocking |
EDIT: this proposal has shifted. See #15735 (comment) below.
Old:
The net/http package needs a way to wait for readability on a TCPConn without actually reading from it. (See #15224)
http://golang.org/cl/22031 added such a mechanism, making Read(0 bytes) do a wait for readability, followed by returning (0, nil). But maybe that is strange. Windows already works like that, though. (See new tests in that CL)
Reconsider this for Go 1.8.
Maybe we could add a new method to TCPConn instead, like WaitRead.
The text was updated successfully, but these errors were encountered: