-
Notifications
You must be signed in to change notification settings - Fork 17.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
proposal: context: new package for standard library #14660
Comments
For others following, there was a substantial discussion on golang-dev about this as it related to As for naming: the only stutter is in the field name in the source. Package users are going to call Looking forward to this happening! |
Thinking about http.Handler, people can alternatively rewrite a different kind of handler that has a different signature for ServeHttp. The advantage is that one can easily add a Context parameter. The top level mux object that encapsulates the whole request dispatching logic can still remain a regular http.Handler (and the context.Background would be instantiated in its ServeHttp(w,r) and passed to the handler ServeWithCtx(ctx,w,r)). Probably better that I upload an example of what I mean. There are also a few things that I am not very fond of. context.TODO is one of them. But I don't know if it is too late to change things or not. |
I think naming it Context in structs is good. There's a stutter, but only for the author of the API. Couldn't be clearer for the users of the API. Would it be possible to write a go vet check for obvious misuse of a Context in a struct? If so, it would be great to roll that out at the same time. Posting a sign next to the pool is good, but a lifeguard is better. As much as I'd love Context to be deeply baked into the standard library, I'd be happy with it simply being added to the standard library to say "this is blessed, find good ways to use it" and later adding the best ideas to the stdlib where they can be and kept in third party packages where they cannot. However, I'm not opposed to any of the suggested integrations. Having the first interface{} parameter to Query be a Context is clever, though, so I'd much rather have the explicit QueryContext. |
This is maybe a bit out of place for this particular issue, but I think it would be worth considering a different Context implementation (strawman benchmark) before it gets included in the stdlib. In that benchmark, I compare a bogus map+lock "caching" context implementation to the default implementation. Even with as few as 10 items, there's a noticeable improvement (%-wise, not in absolute terms. I realize we're talking nanoseconds here) on lookups into the cached version. For contexts deeper than 10 items, there's a noticeable improvement. The construction overhead is similar, but I think with a non-bogus implementation the map version could be even faster for overhead than the linked list one. I think there's an opportunity to have Context be a *struct-with-hidden-fields instead of an interface, which would afford the opportunity to build a faster implementation of it (e.g. by having .WithValue be a struct method instead of a package method that can be smart about chaining, or a .WithValues to avoid the linked-list-building overhead when adding multiple items to the context at the same time) and by possibly adding a .Compact method to compactify an existing Context to make subsequent lookups fast. It's not possible to build these things in a clean fashion with the current interface-based implementation (since the only way for the context to see up the chain is to actually do lookups, and the only way to know the lookup keys is to wait for some user code to provide them). I'm mostly thinking about the pattern of: func MyHTTPHandler(rw ResponseWriter, r *Request) {
c := context.Background()
// WithValue 10-30 services/contextful things here... this number will likely grow
// as context becomes even more prevalently used.
// 100's of lookups into c here... if it were compacted here then all the lookups
// would be O(1).
myActualHandler(c, rw, r)
} Which, I think, is a pretty common pattern (I don't have data on hand for this assumption, however). In general, for most stdlib-type libraries, there would simply be competition for the best implementation. But I think that the community is rallying around a single Context implementation (because it's THE way to build composable/stackable libraries which are loosely coupled). There can't really be an alternate implementation that doesn't stem from the stdlib (or Note that the edit: typo |
How do you expect putting context in http.Request (or any other parameter-ish struct) will interact with adding values (or timeouts or whatever) to the context? I can think of two basic approaches, caller-save and callee-save:
or
Of course, it's fine if both caller and callee save the old value, but it's bad if neither of them does it, each expecting the other to do so. So, should we recommend one and document and promote it up front? |
Traditionally, context objects should carry configuration. The x/net/context package extends context objects' responsibilities by providing constructs for timeout and cancellation signals. In the scope of stamp coupling of request handlers, it is a valuable addition. But I am not sure how it will scale to the rest of the standard library and furthermore whether the context may end up being abused for signaling purposes. The two cases you mention above (exec.Cmd and sql.DB.Query) does this by solely using a context object for handling signals even though there is no necessity for storage. Are you trying to solve the issue of how data is passed among functions or also trying to have primitives around how the entire execution could be signaled from outside? One of the concerns I would add to the list is that the context should be scoped to a function call. "Do not store Contexts inside a struct type; instead, pass a Context explicitly to each function that needs it. The Context should be the first parameter, typically named ctx" (link) If the standard library won't be agreeing with this due to compatibility reasons, don't you think it will create confusion? |
@rakyll, I addressed that final concern in my first post. ("While we've told people...") |
If there's any way to add context to the standard library without making http or net aware of context (can still adjust them to play well with it though), this would be the way to not overwhelm newcomers reading the docs. |
The plan @bradfitz is proposing makes http and net aware of context, but not in a way that requires their use for basic cases. The http package is already very large, but you only need to use a very small fraction of it to achieve "hello world"-esque examples. Adding context to these packages won't harm newcomers, and should make life easier in more complicated/large codebases. |
I mailed out https://golang.org/cl/20346 &https://golang.org/cl/20347 The first copies x/net/context to the standard library. The second reworks x/net/context to just use the standard library's version. |
CL https://golang.org/cl/20346 mentions this issue. |
CL https://golang.org/cl/20347 mentions this issue. |
@kr, making a new request ( |
@riannucci, I think redesigning Context is too big and out of scope. We have tons of code already using the existing Context and we want to make migration easy. |
@bradfitz yeah, I kinda figured as much, but I was hoping that since it's only in x now and not in the stdlib, we still had some API flexibility (e.g. introduce new one to stdlib, deprecate x version, remove x version in 1.8-era or something like that) |
I would love to see support for Context in ctx := context.WithValue(context.TODO, "Title", "Through the Looking Glass")
template.Must(template.Parse(`Title: {{.Title}}`)).Execute(os.Stdout, ctx) would output something like:
If we are worried about accidentally leaking stuff from the Context into a template (because you used a string as a key elsewhere), then I propose adding a special type to distinguish values you want exposed: // in the text/template package
type Expose string // in your program
ctx := context.WithValue(context.TODO, template.Expose("Title"), "Through the Looking Glass")
template.Must(template.Parse(`Title: {{.Title}}`)).Execute(os.Stdout, ctx) This way, you are deliberate about exposing values to the template. |
@chowey: what problem you are trying to solve? If keys have to be dynamic, why not to use a map? |
@bradfitz @riannucci yes, that's my sentiment too. I understand though that x/context is widely used but since it's about to be frozen into the stdlib, I'd rather have the simplest interface possible. Not only for the current users but the ones that have yet to come. I will still understand if it's difficult from the POV of logistics but... |
What do you find 'not simple' about the current net/context API? The most As Brad says though, it's probably a little too late to change anything: On Mon, Mar 14, 2016 at 6:36 AM atdiar notifications@github.com wrote:
|
@elithrar well, the most interesting use is the propagation/monitoring of cancellation signals, not really WithValue (even though it helps for handler chaining). I understand what Brad said. Still I'd rather the decision be clearly examined for the long term. It was in x/context and not the stdlib for a reason. context.TODO was probably supposed to be for static analysis but I think it is not needed. |
I share @atdiar 's concerns. While I understand how golang.org/x/net/context works, it took me multiple reads through the godocs and blog post for it to click. I find the current API kind of confusing. I threw together an alternative implementation (albeit in a rush) with what I believe is a simpler API and easier learning curve: https://github.com/bobziuchkovski/context, godocs at https://godoc.org/github.com/bobziuchkovski/context I realize it's a bit late in the game, but I feel it's at least worth discussing the current x/net/context API before it hits stdlib. |
@bobziuchkovski For posterity, can you update your comment (or add a new one) to clarify the pros/cons of your lib vs. the existing x/net/context? I can see that |
Sure. For TimeRemaining() vs Deadline(), there's more value (for me anyway) in knowing how much time is left vs an expiration date. I can't personally think of a case where I'd want the Adding the I think my biggest concern with the current x/net/context is the CancelFunc stuff and the Background() and TODO() funcs. I "get it" now, but those were the biggest barrier to my understanding of the package. It's far easier for me to comprehend I'm not necessarily putting forth this alternative implementation as the alternative, but rather an alternative to hopefully spark some discussion around the current x/net/context API. |
Thanks Bob. I think that—given the recurring theme of 'net/context not being clear (to address another comment: net/context exists to replace mutex-protected On Mon, Mar 14, 2016 at 9:48 AM Bob Ziuchkovski notifications@github.com
|
@elithrar I definitely agree there. Even without API changes, a lot of my concerns could be addressed via documentation. |
@bradfitz btw, I noticed your note about potentially changing the baseCtx from the server for each request in cabb140 Perhaps the same or related, what do you think about having a root context for the server that is used as the parent for each request context? this way, as the server is shutting down, it can signal to every context it should finish or cancel? |
@pkieltyka, "it should finish" sounds like graceful shutdown (#4674) but semantically, canceling a context is much more abrupt than that. |
@bradfitz yea its true that canceling every in-flight context would be pretty abrupt and what I'm referring to is for purposes of a graceful shutdown. The context tree is the one thing that could actually make it graceful though. Depends how a listener on the ctx.Done() channel considers the signal - should it cancel abruptly or just stop on the next tick? I'm pretty sure in the wild we'll see both cases. hmm.. should there be a ctx.Canceled() channel? |
FWIW, our team has used the cancelation of contexts for graceful shutdown (e.g. wiring SIGTERM -> cancel()). It would be useful to have degrees of cancellation (e.g. STOP EVERYTHING NOW v. Stop when you can). That said, we tend to write crashsafe software, so when we want stop-everything-now, we usually don't want to rely on the application logic to implement that (and we just |
@pkieltyka, @riannucci, changing the context API and/or semantics aren't really in scope for this bug. They've been frozen for some time. I suggest we move any discussion of graceful shutdown to #4674. The answer may involve contexts in some way, but not by canceling some root context. |
CL https://golang.org/cl/22529 mentions this issue. |
Updates #14660 Change-Id: Ifa5c97ba327ad7ceea0a9a252e3dbd9d079dae54 Reviewed-on: https://go-review.googlesource.com/22529 Reviewed-by: Ian Lance Taylor <iant@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
There's a typo @ 2cc27a7#diff-2f929a7660c606fd0c5b268dad1a9bd4R415 waitDone is redefined inside the if block. |
@OneOfOne, https://go-review.googlesource.com/22581. In the future, you can reply to code in Gerrit which we use for our code reviews. |
This is done, so closing. Context lives in package context for now (maybe subject to change yet, but unlikely), and is used by net, net/http, net/http/httptrace, and os/exec for now. The database/sql changes to use Context didn't happen for 1.7. |
Are there some contending alternative names that have been suggested? I am just curious, because I could not think of any. If there was a discussion about this elsewhere, can you point to that; I'd rather not start bikeshedding here. |
Should we update blog.golang.org/context to point users to the
standard context package in Go 1.7+?
|
@minux, no rush. Eventually. Maybe at Go 1.8 time. |
I share you concern about the Now I use below pattern in my codes,
Do you think it could help the looking up performance or not? |
@phuslu instead of guessing, why not write a benchmark? |
@Thomasdezeeuw Thank you. But the more important thing(I think) is, Does this pattern is the RIGHT practice of context.Context ? As you see, the original context.Context is used as a nested structure and seems that each node is immutable(Please correct me if I misunderstand). the original common pattern is,
And the pattern which I used is treat context.Context as a mutable object -- although it may has potential performance improvement.
It smells bad. Hopefully could get comments from you and others. |
@phuslu @riannucci @Thomasdezeeuw, please move discussion of Context.WithValue performance to a new bug. |
Hello!! I think we'll be able to share values between middlewares in the following code by go 1.7. func (a *authHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
user := GetUserFromRequestCookie(r)
ctx := r.Context()
ctx = context.WithValue(ctx, "user", user)
r = r.WithContext(ctx)
a.next.ServeHTTP(w, r) // next handler is ArticleHandle
}
func (a *ArticleHandle) ServeHTTP(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
user := ctx.Value("user").(*User)
} But func (a *authHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
user := UserFromSession(r)
ctx := r.Context()
ctx = context.WithValue(ctx, "user", user)
// r = r.WithContext(ctx)
r.SetContext(ctx)
a.next.ServeHTTP(w, r)
}
func (a *ArticleHandle) ServeHTTP(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
user := ctx.Value("user").(*User)
} Please tell me best way sharing values between middlewares using only net/http and context. I think we should not implement context wrapper like 3rd party framework impl for only sharing values. |
@sona-tar We don't the issue tracker to ask questions. Please see https://golang.org/wiki/Questions for good places to ask. Thanks. |
@Sajmani and I are interested in adding
golang.org/x/net/context
to the standard library, probably as the top-level packagecontext
.(Why not
net/context
? It has little to do with networking, except that people often use it with networking; it doesn't even cross network boundaries on its own without help.)Some places in the standard library which could use it:
Context
field, alongside like the existing (Context.Done-compatible)Request.Cancel
field. It will take precedence overCancel
, and we can make it work for both outbound and inbound requests. For server requests it would useCloseNotifier
's mechanism for c* ancelation. (which for http2 includes the http2-level explicit stream cancelation frame too, also used by gRPC)http.Request
.net.Dialer
also has a Done-compatibleCancel
field, which could be a Context.Context
field to automatically kill+wait the process when a context expires, if the context expires before the child?...interface{}
which can be special-cased to say that if the first parameter is acontext.Context
, it's used for deadlines. Trashy or clever? Maybe both. We could also add parallelDB.QueryContext
which is a bit of a mouthful.Open questions:
io
package in the same release, or later? i.e. do we add*os.File.IO(context.Context) io.ReadWriteCloser
sort of methods, to curry a context intoio
interfaces? Or wait on that. If it'd be nice to push down cancelation into even OS-level file I/O, but we don't even really do that inside Google withIO
-like methods and pervasive Context usage, since we don't really use*os.File
. So I'm inclined to wait on that. Too many operating systems to deal with.Concerns:
The text was updated successfully, but these errors were encountered: