New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
controllers + apiserver: enhance context support #122148
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -18,6 +18,7 @@ package testing | |
|
||
import ( | ||
"context" | ||
"errors" | ||
"fmt" | ||
"net" | ||
"os" | ||
|
@@ -83,13 +84,15 @@ func NewDefaultTestServerOptions() *TestServerInstanceOptions { | |
// files that because Golang testing's call to os.Exit will not give a stop channel go routine | ||
// enough time to remove temporary files. | ||
func StartTestServer(t Logger, _ *TestServerInstanceOptions, customFlags []string, storageConfig *storagebackend.Config) (result TestServer, err error) { | ||
stopCh := make(chan struct{}) | ||
// TODO: this is a candidate for using what is now test/utils/ktesting, | ||
// should that become a staging repo. | ||
ctx, cancel := context.WithCancelCause(context.Background()) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We cannot use There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Add TODO? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Added. |
||
var errCh chan error | ||
tearDown := func() { | ||
// Closing stopCh is stopping apiextensions apiserver and its | ||
// Cancel is stopping apiextensions apiserver and its | ||
// delegates, which itself is cleaning up after itself, | ||
// including shutting down its storage layer. | ||
close(stopCh) | ||
cancel(errors.New("tearing down")) | ||
|
||
// If the apiextensions apiserver was started, let's wait for | ||
// it to shutdown clearly. | ||
|
@@ -166,13 +169,13 @@ func StartTestServer(t Logger, _ *TestServerInstanceOptions, customFlags []strin | |
} | ||
|
||
errCh = make(chan error) | ||
go func(stopCh <-chan struct{}) { | ||
go func() { | ||
defer close(errCh) | ||
|
||
if err := server.GenericAPIServer.PrepareRun().Run(stopCh); err != nil { | ||
if err := server.GenericAPIServer.PrepareRun().RunWithContext(ctx); err != nil { | ||
errCh <- err | ||
} | ||
}(stopCh) | ||
}() | ||
|
||
t.Logf("Waiting for /healthz to be ok...") | ||
|
||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -17,6 +17,8 @@ limitations under the License. | |
package server | ||
|
||
import ( | ||
"context" | ||
"errors" | ||
"fmt" | ||
"io" | ||
"net/http" | ||
|
@@ -42,6 +44,7 @@ import ( | |
"k8s.io/client-go/kubernetes/fake" | ||
"k8s.io/client-go/rest" | ||
"k8s.io/component-base/tracing" | ||
"k8s.io/klog/v2/ktesting" | ||
netutils "k8s.io/utils/net" | ||
) | ||
|
||
|
@@ -79,6 +82,9 @@ func TestAuthorizeClientBearerTokenNoops(t *testing.T) { | |
} | ||
|
||
func TestNewWithDelegate(t *testing.T) { | ||
_, ctx := ktesting.NewTestContext(t) | ||
ctx, cancel := context.WithCancelCause(ctx) | ||
defer cancel(errors.New("test is done")) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. for the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. That would have to happen at the place where some code current reports the We don't take much advantage of this relatively new Go feature yet. I'm adopting it for all new code that I am writing because it is so damn hard to figure out why a context is canceled during debugging. |
||
delegateConfig := NewConfig(codecs) | ||
delegateConfig.ExternalAddress = "192.168.10.4:443" | ||
delegateConfig.PublicAddress = netutils.ParseIPSloppy("192.168.10.4") | ||
|
@@ -136,10 +142,8 @@ func TestNewWithDelegate(t *testing.T) { | |
return nil | ||
}) | ||
|
||
stopCh := make(chan struct{}) | ||
defer close(stopCh) | ||
wrappingServer.PrepareRun() | ||
wrappingServer.RunPostStartHooks(stopCh) | ||
wrappingServer.RunPostStartHooks(ctx) | ||
|
||
server := httptest.NewServer(wrappingServer.Handler) | ||
defer server.Close() | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🤔 – some go proverb: don't store contexts.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I know, but sometimes the alternative is breaking the API badly. I bet that was the reason for cobra's decision to add a command context. As that is now part of their API, we should use it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I understand why they do it. It's always about late initialization of a context that is wired in constructor code. So you have to cut the link and that only work by accessing a stored context.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
question: is the context from the root command automatically propagated to the sub commands? after taking a quick glance, looks like it rests with the author to propagate the
ctx
to the sub commands.Just wondering if
genericapiserver.SetupSignalContext()
can get called more than once for a kube-apiserver processThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I need to check how Cobra handles sub-commands.
Regarding
genericapiserver.SetupSignalContext
that can be called more than once because it usessignal.Notify
under the hood and that can be called more than once:There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh, wait:
https://github.com/kubernetes/apiserver/blob/9b2e61f60b8a7d50ccfb40adad37780cf4038cbf/pkg/server/signal.go#L41
Why? That seems unnecessary.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Perhaps because that code not only maps the signal to the context, it also invokes os.Exit. IMHO that would still be safe to set up more than once, just not desirable.
@stts: you wrote that code, do you remember the rationale?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes. I checked with this patch:
Then in a sub-command like
kubernetes/staging/src/k8s.io/kubectl/pkg/cmd/run/run.go
Line 157 in 5fe1e92
dlv
confirms thatcmd.ctx
is indeed the context prepared bySetupSignalContext
: