Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: Go 2: update context package for Go 2 #28342

Open
ianlancetaylor opened this issue Oct 23, 2018 · 57 comments
Open

proposal: Go 2: update context package for Go 2 #28342

ianlancetaylor opened this issue Oct 23, 2018 · 57 comments

Comments

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Oct 23, 2018

This issue is intended to cover ideas about updating the context package for Go 2.

  • The current context package leads to stuttering in declarations: ctx context.Context.
  • The current Context.WithValue function accepts values of any types, which is easy to misuse by passing, say, a string rather than a value of some package-local type.
  • The name Context is confusing to some people, since the main use of contexts is cancelation of goroutines.
  • Context values are passed everywhere explicitly, which troubles some people. Some explicitness is clearly good, but can we make it simpler?

See also #24050 and #20280.

@ianlancetaylor
Copy link
Contributor Author

@ianlancetaylor ianlancetaylor commented Oct 23, 2018

See also #27982.

Loading

@bcmills
Copy link
Member

@bcmills bcmills commented Oct 23, 2018

I would add to that list:

  • context.WithValue (intentionally) obscures information about the actual inputs required by a function; for example, because those inputs are required by some method of some other parameter. If we're getting real parametricity, it would be preferable to make those requirements explicit at compile time.

Loading

@OneOfOne
Copy link
Contributor

@OneOfOne OneOfOne commented Oct 23, 2018

The name Context is confusing to some people, since the main use of contexts is cancelation of goroutines.

Related #28017

Loading

@josharian
Copy link
Contributor

@josharian josharian commented Oct 24, 2018

I have struggled with performance issues in the past with package context, which were tied somewhat to the API. This led to one hack fix and contemplation of some ugly compiler changes (#18493). I’d like any re-think of context to see how lightweight we can make the package so that it can be used in very fine-grained contexts.

Loading

@mvdan
Copy link
Member

@mvdan mvdan commented Oct 24, 2018

/cc @rogpeppe who I think had some ideas about the design of the API when it was first introduced.

Loading

@ianlancetaylor
Copy link
Contributor Author

@ianlancetaylor ianlancetaylor commented Oct 24, 2018

See also #28279.

Loading

@deanveloper
Copy link

@deanveloper deanveloper commented Oct 26, 2018

Perhaps have go return a one-buffer channel. Closing the channel would cancel the goroutine, and we could have semantics similar to #27982 to check for cancellation. Reading from the channel would read the value that the function returns.

The problem with this, is that there is no way to read multiple return values, which is a pretty big issue. Not quite sure about an elegant way to solve this without having tuple types built-in to the language.

Writing to the channel would unfortunately write to the channel, but if you do stupid things, you should expect stupid behavior. It doesn't have any real use except for breaking your own code. (And we can't have it return a read-only channel if we want to be able to close it)

Context of course would still exist, but it's primary purpose would be for goroutine-local values, rather than cancellation.

Loading

@joonas-fi
Copy link

@joonas-fi joonas-fi commented Nov 30, 2018

I feel that Context only partly solved the problem of cancellation, since there is no facility to wait until the thing we're cancelling, has been cancelled (e.g. wait for the cancelled goroutine to stop). I built a "stopper" for this use case: https://github.com/function61/gokit/blob/dc75639388d554c7f79ca7ec6c967436b6c8301c/stopper/stopper_test.go (disclaimer: I should've made "stopper" a hybrid composed of context - my design is far from good and I plan to improve it)

Loading

@deanveloper
Copy link

@deanveloper deanveloper commented Nov 30, 2018

I like this comment: #21355 (comment)

A somewhat less invasive way to reduce the syntactic overhead of Context would be to make it a built-in type (like error).

That would at least cut the stutter by a third:

-func Foo(ctx context.Context) error {
+func Foo(ctx context) error {

I think my only issue with it is that functions that take contexts also typically have a Context in the function name, ie FooContext(ctx context.Context) error would become FooContext(ctx context) error, and we don't save quite as much as we think we do. Either way cutting off the redundancy is nice.

Loading

@dayfine
Copy link

@dayfine dayfine commented Dec 15, 2018

@deanveloper any example of a FooContext function that actually have 'context' in the name? I haven't seen any of that myself. But context does spread all the way deep into call hierarchy which is sort of horrifying.

Loading

@dayfine
Copy link

@dayfine dayfine commented Dec 15, 2018

Partially copying my reply in #20280

As someone who has been writing Go for two months. I never really understood context, or had to understand it, even though I see it everywhere and use it everywhere, as required by the code standard to pass it down in call hierarchy as the first argument. Today I am refactoring some code that uses it, and wonder if I can get it removed, so I figured out what context is for for the first time.

It is not immediate obvious for a beginner to understand the purpose context is used for, or to understand that IO would like to use context for cancellation, instead of something else, or that context is a primary mechanism of cancellation at all...

It's simply not self-explanatory enough, and I don't think the terminology is consistent for a developer who also deals with concepts with the same name in a different language. e.g. writing C++ at the same time, I would think Context has server-lifetime instead of request-lifetime.

I think that makes it worthwhile for us to carefully explain what it is to help engineers / new learners understand, and make the package itself as clear as possible. e.g. If it is for request-scoped info, why is it package not named request/context, or maybe it should be concurrency/context? Or if it is really used in conjunction with goroutine, how come it is not part of the language feature?

[The part where I don't know what I am talking about]
I feel like some of the concepts were borrowed from the web, where the functionalities, e.g. canceling an asynchronous operation might be realized by a browser API, i.e. to implement the behavior on call stack alone might not be sufficient, and require invention like this which gets a bit confusing. If that's the reality, we should state the facts very clearly and be crystal clear about the limitation and potential confusions.

Loading

@deanveloper
Copy link

@deanveloper deanveloper commented Dec 15, 2018

@deanveloper any example of a FooContext function that actually have 'context' in the name? I haven't seen any of that myself. But context does spread all the way deep into call hierarchy which is sort of horrifying.

Here are a few examples off the top of my head, you can probably make a grep which could find all instances

Edit - Here's a third-party lib that I've used as well https://godoc.org/github.com/nlopes/slack

Loading

@dayfine
Copy link

@dayfine dayfine commented Dec 15, 2018

@deanveloper oh this has to do with providing two versions of everything that either use context or not, as Go doesn't support overloading? Yeh that might need to be solved at a different level :|

Loading

@deanveloper
Copy link

@deanveloper deanveloper commented Dec 15, 2018

I think not supporting overloading is actually a good thing. Makes sure that code stays readable and encourages writing multiple functions for different actions (rather than using something like a boolean flag). Although it does lead to cases like this, unfortunately.

Loading

@joonas-fi
Copy link

@joonas-fi joonas-fi commented Dec 15, 2018

I don't think it has anything to do with overloading, rather I think it is so because the context package was introduced fairly recently, and due to Go's backwards compatibility there has to be separate mechanisms for incrementally improving code (DialContext() as in different function, or a new context field like in net/http). If the context package was there from the start, there would probably be just one net.Dial() that takes context as first arg.

Loading

@deanveloper
Copy link

@deanveloper deanveloper commented Dec 15, 2018

Correct, but if Go did support function overloading, then a Dial function could have been written with a context as the first argument and not broken compatability.

It was a combination of adding features in later as well as having no support for overloading.

Loading

@dayfine
Copy link

@dayfine dayfine commented Dec 15, 2018

@deanveloper or library writers can adopt a currying approach https://golang.org/pkg/net/http/#Request.WithContext

I also meant that having 'Foo' and 'FooContext' doesn't have to do with overloading, but it's just something library writers have to deal with, rather than something that would impact any decision regarding context itself here.

Loading

@creker
Copy link

@creker creker commented Dec 16, 2018

@dayfine

It's simply not self-explanatory enough, and I don't think the terminology is consistent for a developer who also deals with concepts with the same name in a different language. e.g. writing C++ at the same time, I would think Context has server-lifetime instead of request-lifetime.

I agree. I think this is due to context being a combination of two completely separate concepts - WithValue aka the real "context" and cancellation. There're several blog posts written about this problem and the solution I would like Go to pursue is to split these concepts into separate types. Ideally I would like to WithValue to go away completely and leave only cancellation part of the context. After that it can be renamed to clearly state what it's for. We can look at CancellationToken from C# as an example.

Standard library have no use for WithValue at all. It should only accept cancellation tokens. Passing arbitrary values between calls might be useful but it should be limited to outside code. I don't even remember any standard library function that reads values from context. Is there any?

Loading

@acln0
Copy link
Contributor

@acln0 acln0 commented Dec 16, 2018

@creker

I don't even remember any standard library function that reads values from context. Is there any?

net/http is one such package. It inspects client request contexts for net/http/httptrace hooks, and calls them.

Loading

@vearutop
Copy link
Contributor

@vearutop vearutop commented Jan 24, 2019

If we drop WithValue from context, what could be the transport for layers of middlewares?

Loading

@deanveloper
Copy link

@deanveloper deanveloper commented Jan 24, 2019

The point isn't to drop WithValue from context, but rather to split context into two separate ideas.

Loading

@robaho
Copy link

@robaho robaho commented May 30, 2020

@michael-schaller if it is not explicitly passed as a parameter it has the same semantics as TLS/GLS since almost all uses of context - except for cancellation - requires creating a new one to hold the request state (or in the case of pprof labels) - which brings the same name collisions / global namespace issues. But I am willing to be that most framework uses of context.Context probably store a map in the Context, so they only have a single Context instance per routine, and then modify the map, rather than creating a new Context using WithValue - this is the only way to pass information back up to higher-level monitoring code.

For those interested, I suggest looking at ThreadLocal in Java. A difference being that because of generics it is type-safe, but the important thing is that 99+% of all usages are in frameworks or to avoid concurrency penalties when sharing expensive to create objects. Go's context.Context does not address the latter use case (nor really the first very well).

Loading

@michael-schaller
Copy link
Contributor

@michael-schaller michael-schaller commented May 31, 2020

@robaho Thanks for the explanation. I think I understand now what you were hinting at. Let me try to rephrase it in my words. Please correct me if I got any of this wrong (again). ;-)

GLS (if it would exist in Go) would be available to the whole stack of a goroutine and hence it could affect all function calls of that goroutine, similarly to how a global variable could affect all function calls of a process. My proposal is similar as it could affect the whole stack after an annotation has been made. So annotations made in the very beginning of a goroutine would be effectively GLS. To make matters worse annotations that have been inherited by a new goroutine would be a GLS that isn't limited to a single goroutine anymore.

On the other side with context in Go 1 the programmer always has full control over the provided ctx and how it is passed on. However as ctx in Go 1 is typically just passed on as is or is extended/annotated with one of the context.With* functions means that context in Go 1 has a very similar problem as a ctx created for a new goroutine is effectively GLS for this goroutine. To make matters worse a ctx passed on to any new goroutines is also GLS for that goroutine, which again means that this kind of GLS isn't limited to a single goroutine anymore.

Loading

@robaho
Copy link

@robaho robaho commented May 31, 2020

@michael-schaller that is generally it. To be specific, the arguments against the current Context design, is that:

  1. It is an untyped global namespace
  2. No way to propagate up except storing a map in the context at a higher level
  3. No way to cache expensive objects without concurrency access penalties
  4. Documenting Context use is difficult (related to 1)

Even the “always add to the function signature” only serves to create a very poor GLS and lots of noise.

Loading

@robaho
Copy link

@robaho robaho commented May 31, 2020

One other point to add, in the broad case things are more difficult in Go due to the easy concurrency (similar to using multiple worker pools in handling different stages of a single request in Java) so using GLS for “request state” is not trivial either. It may be that a first class “improved Context” designed for this specific case would be a great language feature.

Loading

@creker
Copy link

@creker creker commented May 31, 2020

No way to propagate up except storing a map in the context at a higher level

I don't think context package should make that easier. That's clearly a design flaw with the application, not context package.

so using GLS for “request state” is not trivial either.

That's exactly why context should be passed explicitly. GLS doesn't cover the same use cases that context covers. Context spans logical scopes (which consist of any number of goroutines), not goroutine or function scopes.

Loading

@robaho
Copy link

@robaho robaho commented May 31, 2020

Loading

@mickeyreiss
Copy link
Contributor

@mickeyreiss mickeyreiss commented Jun 13, 2020

Problem:

In large production applications (a web service in my case), it is unnecessarily difficult to diagnose context.Canceled errors.

Even with diligent error wrapping, the end result is often a string, like

handle request: perform business logic: talk to database: context canceled

There are many potential causes for this context cancelation. Some of them are normal operating procedure, and others require intervention. For example, it could be the request is complete, or a custom-built timeout, or perhaps a graceful application shutdown.

In general, if a context wrapped with multiple WithCancels (or WithTimeouts), the cause for context canceled (or deadline exceeded) is ambiguous without additional logging.

Solution:

I would like to propose the addition of explicit errors to supplement context.Canceled and context.DeadlineExceeded.

With errors (nee xerrors), I imagine it would be useful to implement Unwrap, such that the error Is both Canceled and the custom error.

Here's a strawman to demonstrate the concept:

package example

import (
	"context"
	"errors"
	"fmt"
)

var ErrSomethingSpecific = fmt.Errorf("something specific happened")

func Example() {
	ctx, cancel := context.WithCancel(context.Background(), ErrSomethingSpecific)
	cancel()
	fmt.Println(errors.Is(ctx.Err(), context.Canceled))
	fmt.Println(errors.Is(ctx.Err(), ErrSomethingSpecific))

	// Output:
    // true
    // true
}

Loading

@OneOfOne
Copy link
Contributor

@OneOfOne OneOfOne commented Jun 13, 2020

I got frustrated with that as well and have been using https://github.com/OneOfOne/bctx/blob/master/err.go in local and work projects.

Loading

@deanveloper
Copy link

@deanveloper deanveloper commented Sep 21, 2020

For passing data, perhaps it would be better to use a builtin function to access the data. So instead of a global context variable (which would have .WithValue and .Value), there would instead be addcontext(value) and getcontext(Type) (value, bool) builtins. This would reduce a lot of the namespace bloat with xxxContext functions and having context as the first parameter in every function. Contexts that were added in a function call are removed when the function exits. For instance:

type userContext struct {
    currentUser string
}

func main() {
    printUser()
    withUser()
    printUser()
}

func withUser() {
    addcontext(userContext{ "dean" })
    printUser()
}

func printUser() {
    if user, ok := getcontext(userContext); ok {
        fmt.Println("user:", user.currentUser);
    } else {
        fmt.Println("no user")
    }
}

// output:
// no user
// user: dean
// no user

Note that this also makes contexts much more type safe, which is a huge benefit since current contexts have the type safety of a map[interface{}]interface{}.

This still doesn't feel like the best solution. But hopefully this could inspire another idea that's even better. Perhaps requiring removing context explicitly with defer. Or, context is added when calling a function (ie withcontext(context, functioncall())).

===== EDIT =====
I like the withcontext example which I had brought up in the last paragraph, so I thought I would rewrite my last example using withcontext instead of addcontext:

type userContext struct {
    currentUser string
}

func main() {
    printUser()
    withcontext(userContext{ "dean" }, printUser())
    printUser()
}

func printUser() {
    if user, ok := getcontext(userContext); ok {
        fmt.Println("user:", user.currentUser);
    } else {
        fmt.Println("no user")
    }
}

// output:
// no user
// user: dean
// no user

Loading

@deanveloper
Copy link

@deanveloper deanveloper commented Jan 8, 2021

I'd like to write a more formal proposal for what I had written above, although it seems that people don't like it. Some constructive feedback would be much appreciated. In my eyes, using builtin functions and using types as keys accomplishes three things:

  1. Keys are now guaranteed to be unique. In the "context" package, it is even recommended that one already uses custom types for keys to avoid any namespace collision. (granted, these custom types could use enum values as keys which may be more elegant).
  2. Type safety. Because we now know the type of the context, we have type safety over the context's parameters.
  3. "Context bloat" where many functions begin with xxxContext(ctx context.Context, ...), as the contexts are passed through builtins rather than through function parameters.

The goal of what was written above is strictly for the .WithValue side of contexts. This would also be a backwards-compatible change (minus some minor tooling changes to add additional builtin functions). It is likely also possible to add the "cancellation" side of contexts into the withcontext/getcontext proposal as well with a relatively simple package.

Loading

@tv42
Copy link

@tv42 tv42 commented Jan 8, 2021

@deanveloper All attempts at making context be goroutine-local state, or implicitly passed on the stack, suffer from the same issues: There are functions that need to deal with more than one context, either merging or splitting them as needed. Please read the earlier discussion using those terms. And please do write up a proposal that takes into account the whole challenge, if you think your idea is actually different from the above. And I would personally say please also make that a separate issue and link it here. And note that goroutine-local state as a concept has been firmly rejected in the past.

Loading

@creker
Copy link

@creker creker commented Jan 8, 2021

Not only merging or splitting. The problem is implicit passing itself. Go is not that kind of language and people are generally opposed to that (myself included). It's much preferred that context is passed explicitly.

Loading

@deanveloper
Copy link

@deanveloper deanveloper commented Jan 8, 2021

@creker

The problem is implicit passing itself

Explicit passing is part of the problem this discussion exists though - this is what I mean by "context bloat" where we have every function starting with ctx context.Context as the first argument. If I have functions a, b, c, d, and e, where a provides a context and e consumes from it, then b, c, and d all need to have ctx context.Context as arguments, even if they don't use it themselves.

This also makes refactoring difficult. Imagine if the 5 functions above weren't initially intended to need contexts, but then after some later development, it's decided that e needs to consume from a for whatever reason. Now, contexts need to be added to the signatures of the latter 4 functions, even though in reality only 2 functions are even changing (a providing data and e consuming it)

Go is not that kind of language

I beg to differ - Go likes to be explicit until things get too verbose. Specifying every single function with ctx context.Context is distracting and annoying.

Think back to Java/C++ where the type of the variable is always listed with the variable itself. This got distracting, and was needlessly verbose. Both languages have since added features to remove this, and Go of course has := which also removes the needless verbosity of specifying the type of a variable.

The reason that Go is so great (in my opinion) is that it's simple. Go isn't the kind to do everything explicit - Go is the kind to do everything simple. Explicitly usually correlates with simplicity, but I am arguing that it doesn't in this case, in the same way that other features in Go such as variable type inference and garbage collection do.

@tv42

There are functions that need to deal with more than one context, either merging or splitting them as needed

Sorry, I should have been more clear with the example I had shown. This method of using contexts also allows for more than one context at a time, using the type of the context as a key. However my proposal doesn't support having 2 of the same type of context at the same time, but that is relatively easy to implement by simply declaring a new type with the old type as its underlying type.

My apologies, I misunderstood this. This does imply a single context.Context passed per-function, however I think the drawback is worth it. I would also consider these as edge cases personally, and say that if splitting/merging of multiple contexts needs to be done, then using explicit context passing may be better. My proposal targets the main use-case of using contexts, which is passing from a to e without concerning b, c, and d. I don't want to solve more complicated problems like splitting and merging contexts with magical builtins.

Please read the earlier discussion using those terms

I've been pretty involved in the discussion (minus the many comments discussing if the other proposed method was considered GLS or not). This proposal is mostly an addition to the one proposed earlier, although I believe it highlights the benefits and drawbacks a lot better with examples. If you are referencing this comment, I believe that 2 is a non-issue, contexts should be unidirectional. 3 is going to be hard to solve no matter what. However, this solution solves both 1 and 4 perfectly, as it is no longer untyped, and keys are namespaced by package (both of these solve 1), and because types are keys, documentation is easy to write on the provider side, and read on the consumer side (solving 4).

And note that goroutine-local state as a concept has been firmly rejected in the past.

This is passed on the stack the same way that context.Context currently is, except implicitly now instead of explicitly.


I'll write the proposal and link it here. Thank you both for the feedback, I will also be addressing it in the proposal when it is written.

Loading

@creker
Copy link

@creker creker commented Jan 8, 2021

@deanveloper

Explicit passing is part of the problem this discussion exists though

Well, not really. The problem is verbosity and stutter (ctx context.Context is a mouthful) and explicitness is clearly called out as desirable. That part of the issue could be fixed by making context built-in, for example.

This also makes refactoring difficult.

Now imagine that contexts are implicit. Easy refactoring is impossible because context is entirely runtime concept. Compiler can't help you with it. But with explicit contexts adding or removing context argument clearly shows which call sites are affected. Implicit parameters are prone to hidden errors that can only be found at runtime either in tests or in production. That's actually one of the problems with current context implementation - it can carry arbitrary values that, essentially, become implicit function arguments. It's very hard to track down bugs around them.

That's why explicitness is very important. Context clearly demonstrates function behavior. It's main usage is usually cancellation and it's very important that functions can explicitly tell that they support cancellation by accepting context argument. With implicit passing the only way to provide such intent is by documenting it for every function that expects context. Essentially, we're moving one of the function arguments into the comments. And we all know why this is bad.

That brings us to why Go is not that kind of language. To save a few characters we're trading significant advantages that help with code maintainability. Go always puts that above anything else. And I don't think var/auto/:= example is relevant here. It removes verbosity but doesn't have any significant drawbacks because these languages are statically typed.

I would still like to see a more concrete proposal but I just think in its current state it would be heavily downvoted. Any kind of implicitness is a tough sell for Go. There's been too many examples of that.

Loading

@deanveloper
Copy link

@deanveloper deanveloper commented Jan 8, 2021

It's main usage is usually cancellation and it's very important that functions can explicitly tell that they support cancellation by accepting context argument

This is not how a function says that they support cancellation. A function should state that it supports cancellation by documenting that it supports cancellation. The only thing that accepting a context states is that it may take some arbitrary data as input. For instance in the databse/sql package:

// BeginTx starts a transaction.
//
// The provided context is used until the transaction is committed or rolled back.
// If the context is canceled, the sql package will roll back
// the transaction. Tx.Commit will return an error if the context provided to
// BeginTx is canceled.
// ...

Also, perhaps my function doesn't support cancellation on its own, but its callees do. Now, my callees can return a cancellation error which propagates up the stack without the middle-men needing to support cancellation themselves.

Easy refactoring is impossible because context is entirely runtime concept. Compiler can't help you with it.

This point is somewhat fair. But it depends on what you mean by "easy refactoring", which isn't strictly defined.

Easy refactoring could be talking about only needing to change 2 lines in the 2 places that you're concerned about, rather than 9 lines potentially scattered around several files. (9 lines: 2 lines for every function (the arguments and the next function call), minus 1 line because a() doesn't need to change it's signature).

However, easy refactoring could also mean type-safe-refactoring, which allows us to

To save a few characters we're trading significant advantages that help with code maintainability. Go always puts that above anything else.

This isn't about saving a few characters, this is about reducing the amount of bloat on many functions that are written.

And I don't think var/auto/:= example is relevant here. It removes verbosity but doesn't have any significant drawbacks because these languages are statically typed.

It's actually very relevant, it's almost the exact same issue. Languages like Java/C++ took so long to implement var/auto. This isn't because they couldn't (at least in later years), they didn't implement var/auto because they didn't want to. They thought that keeping the type along with the variable was verbose, but that verbosity made things significantly clearer. However after languages like Swift, Go, Kotlin, Rust, and several others, they saw that removing the verbosity of explicitly defining every variable's type actually made code easier to read. My argument is that passing a context into nearly every function is the same way.

I much appreciate that Go is a verbose language. But in some applications, it is extremely useful for contexts to be passed implicitly rather than explicitly.

Loading

@deanveloper
Copy link

@deanveloper deanveloper commented Jan 19, 2021

As I was writing a formal proposal for this, it just kept getting more and more cumbersome. Adding more builtin functions, describing how the contexts get passed, it just gets worse and worse. Ideally the best change (imo) would be to bake cancelling into the language, keep "contextual data" the way it is, and adding some kind of TypeMap to context.Context to help with type-safety and documentation of contextual values. Maybe if context is added as a built-in type, it wouldn't include cancellation, and cancellation could be baked in.

Loading

@Splizard
Copy link

@Splizard Splizard commented Jan 21, 2021

How about stack-local storage with explicit 'context parameters'
When functions want to explicitly handle context, give them a context parameter. Functions without context parameters will recursively pass along any 'context-paramters' onto functions they call.

Parameters are matched by type (to avoid parameter name/position clashes) so there is only one possible context.Context 'context-parameter'. Can easily be overiden, trivial to deal with multiple contexts (as you can still pass them to a function as a normal parameter). Still explicit when it matters. Works with methods and generics. Prevents context poisoning and is flexible to different context types.

//DoSomething has a context parameter.
//any functions that 'DoSomething' calls will transparently & recursively
//pass ctx along to any functions with an explicit context paramter.
func{ctx context.Context} DoSomethingWithContext() {
    DoSomethingWithoutContext() //automatically passes ctx to this function.
    DoAnotherThingWithContext() //automatically passes ctx to this function.
    DoAnotherThingWithContext{context.Background()}() //override the passed context with a different context.
}

func{ctx context.Context} DoAnotherThingWithContext() {}

func DoSomethingWithoutContext() {
    DoAnotherThingWithContext{context.TODO()}() //override the passed context with a different context.
    DoAnotherThingWithContext() //passes the implicit context provided by the caller.
}

context.Value is no longer needed as packages can declare their own private 'context types' and pass them as type-safe 'context paramaters' across API boundaries.
ie.

type myValue string

func{val myValue}  CheckMyValue() {
    fmt.Println(val)
}

//DoSomething could be defined in a different package.
func DoSomething(do func()) {
    do()
}

//Prints Hello World
func main() {
    DoSomething{myValue("Hello World")}(CheckMyValue)
}

Compiler can optimise away the passing of the 'context parameter' when it knows no functions that can handle the parameter are ever called.

Loading

@deanveloper
Copy link

@deanveloper deanveloper commented Jan 21, 2021

I feel like I would like this better if it didn’t mean that Go could have 3 potential parameter lists for a function (after type parameters are added)

edit - also, what happens if the context hadn’t been provided?

Loading

@deanveloper
Copy link

@deanveloper deanveloper commented Jan 21, 2021

I guess I personally don’t see the benefit of adding a language feature for contexts when they aren’t widely used outside of cancellation.

A much better idea (in my eyes) would be to bake cancellation into the language, and improve context.Context with a TypeMap (after generics)

Loading

@Splizard
Copy link

@Splizard Splizard commented Jan 21, 2021

what happens if the context hadn’t been provided?

It is read as an empty value (nil).

bake cancellation into the language

What does this mean? What does this look like?

Loading

@deanveloper
Copy link

@deanveloper deanveloper commented Jan 21, 2021

It is read as an empty value (nil).

How would I differentiate wanting to pass an empty value vs the value not being provided at all? Presumably if I have a type CurrentUserID int, I would want to tell the difference between the ID being 0 and the ID not being provided. The only workaround in this solution seems to be either passing a pointer to an int, or to pass a struct { CurrentUserID int, Ok bool }. Both of these solutions are a bit clunky to me.

What does this mean? What does this look like?

This means to have cancellation as a native feature in Go. I’m not sure what it would look like, I (and hopefully others) should think about that.

Loading

@michael-schaller
Copy link
Contributor

@michael-schaller michael-schaller commented Jan 27, 2021

Cancellation as a native feature in Go could be implemented in many ways.

The simplest form could be like the current context just without the context.WithValue support. But the question is really why we would bother with that as then we might as well just use a channel like we did in the pre-context days to signal cancellation.

A different idea would be to allow go routine management like the OS allows signalling between processes, which would mean that we would have some way to send a termination signal to a go routine from another go routine. Such a signal implementation would be very different fromcontext though as a ctx propagates through the call chain and hence it is easy to cancel everything the ctx propagated to. With signals this would be different as sending a termination signal to a go routine would only affect this go routine unless that signal would automatically also propagate to all go routines that have been spawned from that go routine (similar to signaling process groups on the OS side). Another open question is also how a go routine would handle a termination signal. There could be a runtime function to check if a go routine has received a termination signal or the runtime could cause a panic in the go routine on termination signal.

I'm sure there are even more options on how to implement go routine cancellation natively in Go. The question for me is really if it is worth to implement go routine cancellation as a standalone feature in Go and if that cancellation support would be as versatile and relatively easy to understand as what we currently have with context. Particularly the propagation strictly down the call chain that we currently have with context could be easily lost and IMHO that is one of the greatest strengths of context. Currently it allows to cancel a whole call tree (tree because of go routines) while still leaving the programmer in control of how the context is propagated and how the cancellation is done.

Loading

@DeedleFake
Copy link

@DeedleFake DeedleFake commented Jan 27, 2021

But the question is really why we would bother with that as then we might as well just use a channel like we did in the pre-context days to signal cancellation.

context.Context without context.WithValue() has three main benefits over just using a chan struct{}:

  • Doesn't panic on a secondary cancellation without needing to do anything extra manually.
  • Allows for simple handling of timeouts as well as manual cancellations.
  • Has a tree hierarchy to it that allows a sub-context that can be canceled separately from the parent but also cancels if the parent gets canceled.

The first and second are relatively easy to do manually with a chan struct{} combined with a sync.Once and time.Timer(), but the third is quite a bit more complicated and extremely useful. I think that that alone warrants the existence of context.Context.

Loading

@whitepilledchadcel
Copy link

@whitepilledchadcel whitepilledchadcel commented Mar 17, 2021

  • The current context package leads to stuttering in declarations: ctx context.Context.
  • Context values are passed everywhere explicitly, which troubles some people. Some explicitness is clearly good, but can we make it simpler?

https://golang.org/ref/spec#Function_types

Consider a breaking language change, a shorthand for ParameterDecl. Today we live with

package main

import (
	"context"
	"fmt"
)

type ctx = context.Context

func getname(ctx ctx) (string, bool) {
	u, ok := ctx.Value("namekey").(string)
	return u, ok
}

func main() {
	ctx := context.WithValue(context.Background(), "namekey", "goku")
	u, ok := getname(ctx)
	if !ok {
		panic("no user")
	}
	fmt.Println("user is", u)
}

what if we had

func getname(ctx) (string, bool) {
	u, ok := ctx.Value("namekey").(string)
	return u, ok
}

at our disposal, all else being equal to the first code block?

ParameterDecl  = [ IdentifierList ] [ "..." ] Type .

vs

ParameterDecl  = IdentifierList [ "..." ] Type | identifier . 

Loading

@atdiar
Copy link

@atdiar atdiar commented Mar 17, 2021

For the stuttering, around the time the context library was created, I created a library with similar functionality with a context type called execution.Context

If we make it so that every function on a single goroutine share an implicit execution context, the problem is going to be about how to propagate changes to this execution context down the call chain. For instance if we want to modify the interface by wrapping it.

I suspect that explicitness as is done nowadays, might be the most practical way.

Loading

@deanveloper
Copy link

@deanveloper deanveloper commented Sep 23, 2021

I'm currently working on a proposal:

  1. Allowing a cancel function to be returned by go myFunc()
  2. Adding a new built-in function, cancelled() <-chan struct which allows us to tell when the current goroutine has been cancelled
  3. A new struct, context.Data, which is an immutable datastore very similar to context.Context, and uses a typemap for storage. Unfortunately because type parameters may not exist on methods, it may be a bit strange to use

Would people be interested in this? My main concern is for the cancelled() function. Also I'm probably going to file context.Data as a separate proposal.

Loading

@cameronelliott
Copy link

@cameronelliott cameronelliott commented Sep 23, 2021

@deanveloper Sounds interesting. Would be nice to see your proposal, and how it compares to ctx.Done(), etc.
Maybe you should be considering WaitGroups, and how waiting for a goroutine could be waited on to finish could be intergrated? cancel, done := go foo() so the parent goroutine can wait for children goroutines to complete.
Well, thats just my wild from the cuff ideas. I'm just learning Structured Concurrency for Go, so am interested in seeing proposals and ideas on how to make SC easier under Go.

Loading

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet