Proposal: Use context throughout kubernetes #815

pnasrat · 2014-08-07T01:34:36Z

In an ideal world we need a way to seamlessly carry trace, debugging, cancellation and other data across boundaries - whether that be process, goroutine or RPC.

The code.google.com/p/go.net/context package provides wiring to do this.
References http://blog.golang.org/context

I'd like to wire this into kubernetes, but want blessing before going off and doing it. If you think this would be of value let me know and I'll send a PR.

brendandburns · 2014-08-07T01:58:31Z

I'm definitely intrigued. Some questions:

Can you expand on how you imagine this working inside k8s?

Does this context continue to work if it is serialized into storage and then de-serialized N-minutes later?

Many of the patterns in Kubernetes are of the form:
Handle HTTP request
Write to persistent store
Some other process reads from persistent store
Process sends a new HTTP request

Enabling tracing through that whole thing would be super cool! Is that what you were imagining? Can you stack multiple contexts together?

Thanks in advance!
--brendan

lavalamp · 2014-08-07T05:00:29Z

This is probably a good idea, but it will add complexity, so we should perhaps decide on the scope. I'm curious what @smarterclayton thinks. We've independently added some support for operations and logging associated with a request, so it's not 100% stuff we don't have.

@brendandburns, I am not so sure that we should consider, say, a replication controller's actions as part of the request that added the replication controller to the system. That sounds like an auth/identity/ownership issue, not an RPC tracability issue.

smarterclayton · 2014-08-07T22:16:37Z

A very cool concept, but since many of the transitions occur across HTTP boundaries (with more planned), how would that context propagate? I had imagined that as things like identity propagation became relevant (and potentially projects) there is a domain model object like a "context" (who am i, which resource scope am I in) that would filter through various calls. I could see that associating that with this sort of context also makes sense.

HTTP request trace-ability through any nested calls would be very desirable though.

smarterclayton · 2014-09-27T14:51:30Z

@derekwaynecarr added the first part of this. I'll probably plumb the context through the client. Once we have both of those this can be closed for more specific needs (cancellation, traceability, etc)

thockin · 2014-10-07T17:04:55Z

Status?

derekwaynecarr · 2014-10-07T17:09:43Z

@thockin - I added it to client and server. There are some parts of server not yet completed - specifically minion registry, but I think that is being handled in a separate PR.

smarterclayton · 2014-10-07T17:28:38Z

Also, we aren't conveying a request identifier for traceability through all client calls, which I think in some respects this shou

----- Original Message -----

@thockin - I added it to client and server. There are some parts of server
not yet completed - specifically minion registry, but I think that is being
handled in a separate PR.

Reply to this email directly or view it on GitHub:
#815 (comment)

bgrant0607 · 2014-12-03T15:28:43Z

@derekwaynecarr Is this finished yet? If not, who is working on the final bits?

Cache custom metrics

brendandburns · 2016-08-30T05:31:55Z

go 1.7 introduces context into the HTTP libs as well as the httptrace library, we should re-consider this in light of those additions.

https://golang.org/doc/go1.7#context

smarterclayton · 2016-08-30T14:50:28Z

I agree, we should wire context (as is) into more places. Storage did it for the etcd3 work.

I don't know that we've resolved the "acting-as" wrinkles with controllers in a coherent way where the context would be related to the end user.

bgrant0607 · 2017-03-09T03:53:00Z

There's also OpenTracing.

http://opentracing.io/

ash2k · 2017-04-13T11:17:59Z

It would be useful to add context to all client methods. Also a lot of places use stopCh <-chan struct{} and context could be used there instead.

odeke-em · 2017-07-21T21:09:32Z

How's it going here?

import (
   "context"
)

"context" is now mainstream for Go1.7, Go1.8 and soon Go1.9.

fejta-bot · 2018-01-01T11:34:39Z

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or @fejta.
/lifecycle stale

Automatic merge from submit-queue (batch tested with PRs 59441, 58264, 59287, 59396, 59439). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>. Add context to all relevant cloud APIs **What this PR does / why we need it**: This adds context to all the relevant cloud provider interface signatures. Callers of those APIs are currently satisfied using context.TODO(). There will be follow on PRs to push the context through the stack. **Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*: Fixes #815 **Special notes for your reviewer**: For an idea of the full scope of this change please look at PR #58532. **Release note**: ```release-note Implementers of the cloud provider interface will note the addition of a context to this interface. Trivial code modification will be necessary for a cloud provider to continue to compile. ```

Automatic merge from submit-queue (batch tested with PRs 59441, 58264, 59287, 59396, 59439). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>. Add context to all relevant cloud APIs **What this PR does / why we need it**: This adds context to all the relevant cloud provider interface signatures. Callers of those APIs are currently satisfied using context.TODO(). There will be follow on PRs to push the context through the stack. **Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*: Fixes kubernetes#815 **Special notes for your reviewer**: For an idea of the full scope of this change please look at PR kubernetes#58532. **Release note**: ```release-note Implementers of the cloud provider interface will note the addition of a context to this interface. Trivial code modification will be necessary for a cloud provider to continue to compile. ```

@sttts

Automatic merge from submit-queue (batch tested with PRs 60012, 63692, 63977, 63960, 64008). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>. Use Dial with context **What this PR does / why we need it**: `net/http/Transport.Dial` field is deprecated: ```go // DialContext specifies the dial function for creating unencrypted TCP connections. // If DialContext is nil (and the deprecated Dial below is also nil), // then the transport dials using package net. DialContext func(ctx context.Context, network, addr string) (net.Conn, error) // Dial specifies the dial function for creating unencrypted TCP connections. // // Deprecated: Use DialContext instead, which allows the transport // to cancel dials as soon as they are no longer needed. // If both are set, DialContext takes priority. Dial func(network, addr string) (net.Conn, error) ``` This PR switches all `Dial` usages to `DialContext`. Fixes #63455. **Special notes for your reviewer**: Also related: #59287 #58532 #815 kubernetes/community#1166 #58677 #57932 **Release note**: ```release-note HTTP transport now uses `context.Context` to cancel dial operations. k8s.io/client-go/transport/Config struct has been updated to accept a function with a `context.Context` parameter. This is a breaking change if you use this field in your code. ``` /sig api-machinery /kind enhancement /cc @sttts

@sttts

Automatic merge from submit-queue (batch tested with PRs 60012, 63692, 63977, 63960, 64008). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>. Use Dial with context **What this PR does / why we need it**: `net/http/Transport.Dial` field is deprecated: ```go // DialContext specifies the dial function for creating unencrypted TCP connections. // If DialContext is nil (and the deprecated Dial below is also nil), // then the transport dials using package net. DialContext func(ctx context.Context, network, addr string) (net.Conn, error) // Dial specifies the dial function for creating unencrypted TCP connections. // // Deprecated: Use DialContext instead, which allows the transport // to cancel dials as soon as they are no longer needed. // If both are set, DialContext takes priority. Dial func(network, addr string) (net.Conn, error) ``` This PR switches all `Dial` usages to `DialContext`. Fixes #63455. **Special notes for your reviewer**: Also related: kubernetes/kubernetes#59287 kubernetes/kubernetes#58532 kubernetes/kubernetes#815 kubernetes/community#1166 kubernetes/kubernetes#58677 kubernetes/kubernetes#57932 **Release note**: ```release-note HTTP transport now uses `context.Context` to cancel dial operations. k8s.io/client-go/transport/Config struct has been updated to accept a function with a `context.Context` parameter. This is a breaking change if you use this field in your code. ``` /sig api-machinery /kind enhancement /cc @sttts Kubernetes-commit: ddf551c24b7d88454f8332ce6855e53281440958

@sttts

Automatic merge from submit-queue (batch tested with PRs 60012, 63692, 63977, 63960, 64008). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>. Use Dial with context **What this PR does / why we need it**: `net/http/Transport.Dial` field is deprecated: ```go // DialContext specifies the dial function for creating unencrypted TCP connections. // If DialContext is nil (and the deprecated Dial below is also nil), // then the transport dials using package net. DialContext func(ctx context.Context, network, addr string) (net.Conn, error) // Dial specifies the dial function for creating unencrypted TCP connections. // // Deprecated: Use DialContext instead, which allows the transport // to cancel dials as soon as they are no longer needed. // If both are set, DialContext takes priority. Dial func(network, addr string) (net.Conn, error) ``` This PR switches all `Dial` usages to `DialContext`. Fixes #63455. **Special notes for your reviewer**: Also related: kubernetes/kubernetes#59287 kubernetes/kubernetes#58532 kubernetes/kubernetes#815 kubernetes/community#1166 kubernetes/kubernetes#58677 kubernetes/kubernetes#57932 **Release note**: ```release-note HTTP transport now uses `context.Context` to cancel dial operations. k8s.io/client-go/transport/Config struct has been updated to accept a function with a `context.Context` parameter. This is a breaking change if you use this field in your code. ``` /sig api-machinery /kind enhancement /cc @sttts Kubernetes-commit: ddf551c24b7d88454f8332ce6855e53281440958

@sttts

Automatic merge from submit-queue (batch tested with PRs 60012, 63692, 63977, 63960, 64008). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>. Use Dial with context **What this PR does / why we need it**: `net/http/Transport.Dial` field is deprecated: ```go // DialContext specifies the dial function for creating unencrypted TCP connections. // If DialContext is nil (and the deprecated Dial below is also nil), // then the transport dials using package net. DialContext func(ctx context.Context, network, addr string) (net.Conn, error) // Dial specifies the dial function for creating unencrypted TCP connections. // // Deprecated: Use DialContext instead, which allows the transport // to cancel dials as soon as they are no longer needed. // If both are set, DialContext takes priority. Dial func(network, addr string) (net.Conn, error) ``` This PR switches all `Dial` usages to `DialContext`. Fixes #63455. **Special notes for your reviewer**: Also related: kubernetes/kubernetes#59287 kubernetes/kubernetes#58532 kubernetes/kubernetes#815 kubernetes/community#1166 kubernetes/kubernetes#58677 kubernetes/kubernetes#57932 **Release note**: ```release-note HTTP transport now uses `context.Context` to cancel dial operations. k8s.io/client-go/transport/Config struct has been updated to accept a function with a `context.Context` parameter. This is a breaking change if you use this field in your code. ``` /sig api-machinery /kind enhancement /cc @sttts Kubernetes-commit: ddf551c24b7d88454f8332ce6855e53281440958

@sttts

Automatic merge from submit-queue (batch tested with PRs 60012, 63692, 63977, 63960, 64008). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>. Use Dial with context **What this PR does / why we need it**: `net/http/Transport.Dial` field is deprecated: ```go // DialContext specifies the dial function for creating unencrypted TCP connections. // If DialContext is nil (and the deprecated Dial below is also nil), // then the transport dials using package net. DialContext func(ctx context.Context, network, addr string) (net.Conn, error) // Dial specifies the dial function for creating unencrypted TCP connections. // // Deprecated: Use DialContext instead, which allows the transport // to cancel dials as soon as they are no longer needed. // If both are set, DialContext takes priority. Dial func(network, addr string) (net.Conn, error) ``` This PR switches all `Dial` usages to `DialContext`. Fixes #63455. **Special notes for your reviewer**: Also related: kubernetes/kubernetes#59287 kubernetes/kubernetes#58532 kubernetes/kubernetes#815 kubernetes/community#1166 kubernetes/kubernetes#58677 kubernetes/kubernetes#57932 **Release note**: ```release-note HTTP transport now uses `context.Context` to cancel dial operations. k8s.io/client-go/transport/Config struct has been updated to accept a function with a `context.Context` parameter. This is a breaking change if you use this field in your code. ``` /sig api-machinery /kind enhancement /cc @sttts Kubernetes-commit: ddf551c24b7d88454f8332ce6855e53281440958

Remove NEXT_KEP_NUMBER from template

cheftako · 2019-10-14T01:21:35Z

/remove-lifecycle rotten

cheftako · 2019-10-14T01:21:49Z

/lifecycle frozen

cheftako · 2020-08-10T17:35:09Z

/reopen

cheftako · 2020-08-10T17:35:17Z

/open

k8s-ci-robot · 2020-08-10T17:35:23Z

@cheftako: Reopened this issue.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

MadhavJivrajani · 2021-06-29T16:05:30Z

/remove-kind design
/kind feature

kind/design will soon be removed from k/k in favor of kind/feature. Relevant discussion can be found here: kubernetes/community#5641

…rry-pick-804-to-release-4.8 [release-4.8] Bug 1973580: only chown if non-windows machine with projected volumes

@sttts

Automatic merge from submit-queue (batch tested with PRs 60012, 63692, 63977, 63960, 64008). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>. Use Dial with context **What this PR does / why we need it**: `net/http/Transport.Dial` field is deprecated: ```go // DialContext specifies the dial function for creating unencrypted TCP connections. // If DialContext is nil (and the deprecated Dial below is also nil), // then the transport dials using package net. DialContext func(ctx context.Context, network, addr string) (net.Conn, error) // Dial specifies the dial function for creating unencrypted TCP connections. // // Deprecated: Use DialContext instead, which allows the transport // to cancel dials as soon as they are no longer needed. // If both are set, DialContext takes priority. Dial func(network, addr string) (net.Conn, error) ``` This PR switches all `Dial` usages to `DialContext`. Fixes #63455. **Special notes for your reviewer**: Also related: kubernetes/kubernetes#59287 kubernetes/kubernetes#58532 kubernetes/kubernetes#815 kubernetes/community#1166 kubernetes/kubernetes#58677 kubernetes/kubernetes#57932 **Release note**: ```release-note HTTP transport now uses `context.Context` to cancel dial operations. k8s.io/client-go/transport/Config struct has been updated to accept a function with a `context.Context` parameter. This is a breaking change if you use this field in your code. ``` /sig api-machinery /kind enhancement /cc @sttts Kubernetes-commit: ddf551c24b7d88454f8332ce6855e53281440958

thockin · 2023-01-24T00:55:25Z

I'm not sure that keeping this open as an issue is useful any more?

Move glog/klog logging to klog/v2

brendandburns added design labels Aug 7, 2014

bgrant0607 added the priority/backlog Higher priority than priority/awaiting-more-evidence. label Dec 3, 2014

bgrant0607 mentioned this issue Jan 28, 2015

WIP - Add trace header for all incoming requests #3826

Closed

davidopp added the team/master label Feb 8, 2015

bgrant0607 added priority/awaiting-more-evidence Lowest priority. Possibly useful, but not yet enough support to actually get it done. status/help-wanted and removed priority/backlog Higher priority than priority/awaiting-more-evidence. labels Feb 28, 2015

mikedanese added sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. and removed team/master labels Aug 20, 2015

vishh pushed a commit to vishh/kubernetes that referenced this issue Apr 6, 2016

Merge pull request kubernetes#815 from anushree-n/cache

649fac9

Cache custom metrics

smarterclayton mentioned this issue May 29, 2016

Consider adding tracing support for OpenTracing (or similar) #26507

Closed

bgrant0607 removed the help-wanted label Aug 30, 2016

bgrant0607 added the triaged label Mar 9, 2017

ash2k mentioned this issue Jun 6, 2017

Shared Informer Run blocks until all goroutines finish #46094

Merged

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jan 1, 2018

ash2k mentioned this issue Feb 17, 2018

Use Dial with context #60012

Merged

Monkeyanator mentioned this issue Oct 3, 2018

OpenCensus tracing support added to Kubelet Monkeyanator/kubernetes#1

Open

seans3 pushed a commit to seans3/kubernetes that referenced this issue Apr 10, 2019

Merge pull request kubernetes#815 from liggitt/next-kep-number

1aca3c4

Remove NEXT_KEP_NUMBER from template

cheftako mentioned this issue Oct 14, 2019

Push context up to cloud node controller. #83872

Merged

k8s-ci-robot removed the lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. label Oct 14, 2019

k8s-ci-robot added the lifecycle/frozen Indicates that an issue or PR should not be auto-closed due to staleness. label Oct 14, 2019

bboreham mentioned this issue Jun 24, 2020

Update to v1alpha3 weaveworks/wksctl#172

Merged

k8s-ci-robot reopened this Aug 10, 2020

k8s-ci-robot added kind/feature Categorizes issue or PR as related to a new feature. and removed kind/design Categorizes issue or PR as related to design. labels Jun 29, 2021

stlaz pushed a commit to stlaz/kubernetes that referenced this issue Jul 1, 2021

Merge pull request kubernetes#815 from openshift-cherrypick-robot/che…

766a5fe

…rry-pick-804-to-release-4.8 [release-4.8] Bug 1973580: only chown if non-windows machine with projected volumes

thockin closed this as completed Jan 24, 2023

linxiulei pushed a commit to linxiulei/kubernetes that referenced this issue Jan 18, 2024

Merge pull request kubernetes#815 from hakman/klogv2

fb49856

Move glog/klog logging to klog/v2

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Proposal: Use context throughout kubernetes #815

Proposal: Use context throughout kubernetes #815

pnasrat commented Aug 7, 2014

brendandburns commented Aug 7, 2014

lavalamp commented Aug 7, 2014

smarterclayton commented Aug 7, 2014

smarterclayton commented Sep 27, 2014

thockin commented Oct 7, 2014

derekwaynecarr commented Oct 7, 2014

smarterclayton commented Oct 7, 2014

bgrant0607 commented Dec 3, 2014

brendandburns commented Aug 30, 2016

smarterclayton commented Aug 30, 2016

bgrant0607 commented Mar 9, 2017

ash2k commented Apr 13, 2017

odeke-em commented Jul 21, 2017

fejta-bot commented Jan 1, 2018

cheftako commented Oct 14, 2019

cheftako commented Oct 14, 2019

cheftako commented Aug 10, 2020

cheftako commented Aug 10, 2020

k8s-ci-robot commented Aug 10, 2020

MadhavJivrajani commented Jun 29, 2021

thockin commented Jan 24, 2023

Proposal: Use context throughout kubernetes #815

Proposal: Use context throughout kubernetes #815

Comments

pnasrat commented Aug 7, 2014

brendandburns commented Aug 7, 2014

lavalamp commented Aug 7, 2014

smarterclayton commented Aug 7, 2014

smarterclayton commented Sep 27, 2014

thockin commented Oct 7, 2014

derekwaynecarr commented Oct 7, 2014

smarterclayton commented Oct 7, 2014

bgrant0607 commented Dec 3, 2014

brendandburns commented Aug 30, 2016

smarterclayton commented Aug 30, 2016

bgrant0607 commented Mar 9, 2017

ash2k commented Apr 13, 2017

odeke-em commented Jul 21, 2017

fejta-bot commented Jan 1, 2018

cheftako commented Oct 14, 2019

cheftako commented Oct 14, 2019

cheftako commented Aug 10, 2020

cheftako commented Aug 10, 2020

k8s-ci-robot commented Aug 10, 2020

MadhavJivrajani commented Jun 29, 2021

thockin commented Jan 24, 2023