Add flags to allow tuning k8s client throttling. #192
Conversation
|
Hi @jkh52. Thanks for your PR. I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
k8s-ci-robot
added
needs-ok-to-test
|
/ok-to-test |
k8s-ci-robot
added
ok-to-test
|
Thanks. |
|
/retest |
|
/assign @caesarxuchao |
|
/lgtm |
k8s-ci-robot
added
the
lgtm
cmd/server/main.go
Outdated
| @@ -154,6 +158,8 @@ func (o *ProxyRunOptions) Flags() *pflag.FlagSet { | |||
| flags.StringVar(&o.agentNamespace, "agent-namespace", o.agentNamespace, "Expected agent's namespace during agent authentication (used with agent-service-account, authentication-audience, kubeconfig).") | |||
| flags.StringVar(&o.agentServiceAccount, "agent-service-account", o.agentServiceAccount, "Expected agent's service account during agent authentication (used with agent-namespace, authentication-audience, kubeconfig).") | |||
| flags.StringVar(&o.kubeconfigPath, "kubeconfig", o.kubeconfigPath, "absolute path to the kubeconfig file (used with agent-namespace, agent-service-account, authentication-audience).") | |||
| flags.Float32Var(&o.kubeconfigQPS, "kubeconfig-qps", o.kubeconfigQPS, "Maximum client QPS (used with authentication).") | |||
There was a problem hiding this comment.
nit: there are multiple authentications in the system. Perhaps "proxy server uses this client to authenticate agent tokens" to be more specific.
|
A nit. LGTM otherwise. |
k8s-ci-robot
removed
the
lgtm
|
/lgtm |
k8s-ci-robot
added
the
lgtm
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: caesarxuchao, jkh52 The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
k8s-ci-robot
added
the
approved
I have seen that if enough agents exist, and one of a set of masters is unavailable (so every agent is sync-ing), proxy server goroutines grows unbounded. With pprof I find that client-go throttling (of token reviews) is where they are blocked.
In my repro, if I replace the client-go defaults (5 qps, 10 burst) with higher numbers, it mitigates the issue.