-
Notifications
You must be signed in to change notification settings - Fork 17.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
proposal: io/v2: add Context parameter to Reader, etc. #20280
Comments
In addition to all of the Reader-like and Writer-like interfaces, we should also do this to It's less obvious whether this should be done to |
Sorry to hijack, but if we get to this point, then |
@rasky, let's file a seperate issue for Context being part of the language. Other than needing to do extra typing to add the argument for the context, there is not that much complexity. Compute-only Readers like compression doesn't even need to inspect the context. All it needs to do is plumb it through to the underlying io.Reader. This does not seem particularly complicated. |
It's real simple to make an |
I don't bother be rude: the author is network monkey. And I believe the problem of goroutine cancellation must be solved via the runtime tricks, because it only happened as a consequence of mismatch between synchronous representation of asynchronous environment, so let they introduce another tool to access the functionality that is out of scope now due to the mismatch. |
@dsnet and I came up with a way this could be introduced gradually. Instead of modifying the interfaces directly, taking the database/sql approach: package ctxio // package "io/ctxio"
import (
"context"
"io"
)
type Reader interface {
ReadCtx(context.Context, []byte) (int, error)
}
// BindReader returns an io.Reader that reads from r using the ctx Context.
// (Name is bikeshed-able.)
func BindReader(ctx context.Context, r Reader) io.Reader
// PromoteReader returns a Reader that reads from r.
// It will ignore any Context passed to the Read calls.
// (Name is bikeshed-able.)
func PromoteReader(r io.Reader) Reader
// and so on... By having this be a different interface, existing concrete types (like @dchapes: Yes, I have done this before. Where @sirkon raises a good point which would be the logical follow-on: it would be wonderful if |
Try to read about blocking and non-blocking IO at least at the API level. TCPConn's socket is already non-blocking in Go and it is only needed to "just" add another coupled file descriptor (timer) into the event loop to control exit. "Just" because it is yet another syscall and syscalls are expensive. On files: asynchronous file IO don't work well in Linux and turn to be blocking for heavy loads. So, you need to launch thread per file to guarantee asynchronous file reads/writes (+ scheduling thread, although it is already there). These were just technical remarks. And it, finally, the idea stinks. It just looks ugly, it is the shame you don't realize this yourself. |
@sirkon, thank you for your concerns about syscall efficiency. However, saying "the idea stinks. It just looks ugly, it is the shame you don't realize this yourself" doesn't help the discussion. I think you may be failing to recognize the problem that this issue is trying to solve. These issues are intended to be a place to discuss ideas in a civil manner. The issue at hand is not just "asynchronous vs synchronous", but one with regards to plumbing context for cancelation signal, which (although often canceled asynchronously) and also value propagation are still different issues than "asynchronous vs synchronous" calling of If you have the "best possible solution" in mind, perhaps you would like to elaborate your ideas? |
I don't have any idea on the issue other than understanding this proposal is stupid. By points:
Back to the proposal:
|
@sirkon, I'm going to echo @dsnet's remarks here. Your responses are uncivil (I would like to point you to our Code of Conduct) and are commenting on the basis of "ugliness" rather than giving concrete technical objections. I understand that an Experience Report will likely clear up some of the background on what problems this is trying to solve. I will try to write up something soon. As for the objection to "introduce concept into domain when not all items of it actually support it", for the implementers of an io interface that don't require the Context, they can simply ignore the parameter, just as any function that takes in a Context that does not need it does. However, consider Readers and Writers that need a Context: there isn't as easy of a workaround. @dchapes's solution works in a number of cases involving deadlines, but it does not work in cases involving Context values. |
For the record, this isn't a full-fledged proposal. I opened this issue since I feel this is something that should be considered for Go 2. It may be accepted; it may be rejected. The Go ecosystem is still finding its way on how and when to use Context. Networking and I/O are places where Context comes up frequently, so it's something that we may want to consider in the future. Once we get farther along and the problem space is more agreed upon, there may be better ways of achieving the same goals. |
I've written an experience report/blog post detailing the background for this proposal: Canceling I/O in Go Cap’n Proto. Alternatives welcome. |
Real network monkeys know that Go also runs on modern systems. |
I think this proposal is asking for something deeper than changing As @dchapes says above, it is possible to build an So I think we can set aside the idea of passing a Can we devise a mechanism for associating a |
On POSIX systems, it would be great if The other part, unblocking a call, is very specific to the file descriptor and the function being used... So far I only had to unblock |
Associating a I created this (incomplete) hack for testing my own purposes that may or may not be useful for reference. Ignoring the implementation, the API at least served my purposes. https://gitlab.com/streamy/concon |
@Julio-Guerra |
@dchapes The proposal you made only works to some extent. For example, if we call Although this example is simple and can be avoided, I have seen more intricate cases with similar ideas which run into the same trouble. |
This is essentially a copy-paste implementation of the ideas presented in this comment here: golang/go#20280 (comment) It fixes #775 and helps with the issue described in #793 (comment). Not sure if it has unintended side-effects.
This is essentially a copy-paste implementation of the ideas presented in this comment here: golang/go#20280 (comment) It fixes #775 and helps with the issue described in #793 (comment). Not sure if it has unintended side-effects.
* Cancel reading from stdin on Ctrl-C This is essentially a copy-paste implementation of the ideas presented in this comment here: golang/go#20280 (comment) It fixes #775 and helps with the issue described in #793 (comment). Not sure if it has unintended side-effects. * Add comments and remove interface * update changelog Co-authored-by: BolajiOlajide <25608335+BolajiOlajide@users.noreply.github.com>
* Cancel reading from stdin on Ctrl-C This is essentially a copy-paste implementation of the ideas presented in this comment here: golang/go#20280 (comment) It fixes #775 and helps with the issue described in #793 (comment). Not sure if it has unintended side-effects. * Add comments and remove interface * update changelog Co-authored-by: BolajiOlajide <25608335+BolajiOlajide@users.noreply.github.com>
In golang, we design interfaces after structs. That means at the beginning, the real logic is not clear, we can use structs to implement it and finally use interface to reuse it in the future. Therefore, whether In conclusion, I don't think we should redesign std interfaces. It's not necessary. |
In lieu of having this feature upstream, and not seeing a complete proof of concept, I've implemented a cancellable password reader. Of course if a signal kills the process, this can leave your terminal in a bad state and you'd need to run https://github.com/purpleidea/mgmt/tree/master/util/password If anyone has improvements, please send a patch! I'd comment in #24842 but it's locked. HTH |
What about a contexual interface? type Contextual[T any] interface {
Context(context.Context) T
}
type contextualReader struct {
ctx context.Context
reader io.Reader
}
func (r *contextualReader) Context(ctx context.Context) io.Reader {...}
func (r *contextualReader) Read(b []byte) (n int, err error) {...}
func logic(ctxReader Contextual[io.Reader]) {
//...ctx...
ctxReader.Context(ctx).Read(buf)
} This interface force users to pass ctx before invoking any other methods. |
Maybe I am missing something, but this doesn't look that hard. I recently wrote a bunch of asynchronous cancellable functions that read from a |
Related to #18507, it would be nice if the
io
interface methods all took inContext
as their first parameter. This allows a cancellation signal to be attached directly to the I/O calls that are using them, instead of requiring an out-of-band method, as innet.Conn
.This is obviously a backward-incompatible change and would have broad impact on the ecosystem. I wanted to open this as a proposal for consideration in Go 2.
EDIT: See below for the more backward-compatible proposal
EDIT 2: I've written an experience report/blog post detailing the background for this proposal: Canceling I/O in Go Cap’n Proto. Feedback and alternatives welcome.
The text was updated successfully, but these errors were encountered: