[WIP]When kubelet enable rotate-certificates and bootstrap, request CSR in foreground

[chenk008]

What type of PR is this?

/kind feature

What this PR does / why we need it:

When kubelet enable rotate-certificates and bootstrap, request CSR in foreground. It can help to reduce the uncertainty of node register time.

Which issue(s) this PR fixes:

Fixes #112170

Special notes for your reviewer:

Now when kubelet enable rotate-certificates and bootstrap-kubeconfig, it will request CSR in background. With the delayed CSR approval, kubelet use the bootstrap token to register, but the bootstrap token has no permission. It will need wait 10s to change client transport tls config.

Requesting CSR in foreground, even the CSR delayed a seconds, the kubelet will register successfully Immediately after the CSR approved. It don't need to wait 10s for the certificate check.

Does this PR introduce a user-facing change?

NONE

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: chenk008 (2b193d2663491a3949c5f749460fcdce6c6458d9)

[k8s-ci-robot]

@chenk008: This issue is currently awaiting triage.

If a SIG or subproject determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[k8s-ci-robot]

Hi @chenk008. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: chenk008
Once this PR has been reviewed and has the lgtm label, please assign mikedanese for approval by writing /assign @mikedanese in a comment. For more information see:The Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• cmd/kubelet/OWNERS
• pkg/kubelet/certificate/OWNERS
• staging/src/k8s.io/client-go/util/certificate/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[leilajal]

/remove-sig api-machinery

[liggitt]

Requesting CSR in foreground, even the CSR delayed a seconds, the kubelet will register successfully Immediately after the CSR approved. It don't need to wait 10s for the certificate check.

It's unclear to me what the implications of blocking on this are for running static pods. I know putting this in the background was explicitly done so that static pods would not be impacted by CSR requests. xref #69890

/cc @smarterclayton

/hold

[chenk008]

@liggitt
Sorry, I just learned the background are for running static pods.

In my scenario:

1. My cluster enableS node autoscale, we hope the node can be ready in 10 seconds after the kubelet process started.
2. The components (e.g. CNI, kube-proxy) are deployed as daemonset,
3. The node ready condition depends on CNI pod.
4. We want the kubelet finish registration as soon as possible, so that these components pods can be created. We expect the CNI pod to be created in 3 seconds.

For now, it will request CSR in background, and check certificates every 10 seconds. https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/certificate/transport.go#L57. Once the CSR delayed, it has to wait for the next cycle, it is too long in our scenario.

How about the following options?

1. Adjust the certificates check period to 1 second. In most cases, the CSR will be approved in 1 second.
2. Add other mechanisms to watch the certificates change (i.e. file inotify)

WDYT?

[liggitt]

is the issue that this line allows successfully establishing connections with no certificate while waiting for the initial certificate, and those connections persist up to 10 seconds once the initial CSR is approved/signed/issued?

kubernetes/pkg/kubelet/certificate/transport.go

Lines 92 to 94 in 7cba7e6

	if cert == nil {
	return &tls.Certificate{Certificate: nil}, nil
	}

It seems reasonable to check more frequently until we get non-null initial certificate... opened #114367 as a possible example of that

[dims]

This PR has the label work-in-progress, please revisit to see if you still need this, please close if not

[enj]

/close
In favor of #114367

[k8s-ci-robot]

@enj: Closed this PR.

In response to this:

/close
In favor of #114367

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.