feat(anc): use gRPC as the check-hotfix live-patching transport - #9050
feat(anc): use gRPC as the check-hotfix live-patching transport#9050Devinwong wants to merge 1 commit into
Conversation
|
The latest Buf updates on your PR. Results from workflow Buf CI / buf (pull_request).
|
93a27f7 to
615f423
Compare
91ba500 to
227948c
Compare
Windows Unit Test Results 3 files 12 suites 38s ⏱️ Results for commit c59d8dc. ♻️ This comment has been updated with latest results. |
c024ebf to
266d036
Compare
266d036 to
b4f6df3
Compare
b4f6df3 to
df712e0
Compare
df712e0 to
e0e8bd1
Compare
e0e8bd1 to
be7f328
Compare
be7f328 to
84b0eb3
Compare
84b0eb3 to
5207157
Compare
5207157 to
6dd0bb4
Compare
…ng contract Switch the check-hotfix subcommand's live-patching-service fetch from the HTTP-over-SNI placeholder (which never ran against a real server) to the gRPC GetComponentConfig contract. Rather than define a second copy of that contract, reuse the shared akslivepatching.v1.LivePatchingService generated stubs from the aks-live-patching module (the agreed cross-team contract), wired in via a local replace directive. The transport dials the apiserver front on the cluster FQDN with TLS verified against the on-disk cluster CA and SNI pinned to the live-patching host, carries the IMDS attested-data document in gRPC request metadata, and maps gRPC status codes onto the existing benign-vs-fatal taxonomy. check-hotfix remains fail-open (always exits 0): until a real gRPC endpoint serves, it is a no-op on every node that stages nothing and never blocks provisioning. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: ed2dd633-00bb-4b3b-885b-9741b917a0a0
6dd0bb4 to
c59d8dc
Compare
| MinVersion: tls.VersionTLS12, | ||
| RootCAs: pool, | ||
| NextProtos: []string{lpsALPNProto}, | ||
| InsecureSkipVerify: true, // nolint:gosec // hostname check disabled on purpose; chain verified below |
AgentBaker Linux gate detectiveRun: 174966791 | Failed job: Run AgentBaker E2E | PR: #9050 (draft/open) TL;DR: E2E test Test_Ubuntu2604Minimal_DisableKubeletServingCertificateRotationWithTags_CustomKubeletConfig failed VM validation because grub2-common.service unexpectedly entered a failed state on the test node (Ubuntu 26.04 Minimal). 289 of 290 tests passed. Likely cause: Transient Ubuntu 26.04 VM infrastructure issue. grub2-common.service is a GRUB configuration service that occasionally fails on Ubuntu 26.04 nodes during first-boot or systemd initialization. The same signature was previously observed in build 174811907 (PR #8896) affecting a different scenario (Test_Ubuntu2604Gen2_McrChinaCloud). This PR (#9050) is an ANC gRPC hotfix transport change with no grub2/bootloader changes. Classification: E2E VM validation / transient infrastructure — not a product regression caused by this PR. Confidence: High — single test failure in 290; grub2-common.service is unrelated to ANC gRPC transport; identical service failure pattern seen in prior unrelated PRs. Strongest alternative: PR-introduced regression affecting Ubuntu 26.04 minimal provisioning. Ruled out: the PR changes ANC gRPC transport (unrelated to grub2), and the failure pattern is seen across unrelated PRs. Recommended action: Rerun the gate; if this test passes on rerun, it confirms transient infra. If it recurs, investigate grub2-common.service on Ubuntu 26.04 Minimal images. Wiki signature: linux-e2e-ubuntu2604-gen2-mcrchinacloud-grub2-common-service-failed (count: 2) |
What
Switch the aks-node-controller
check-hotfixfetch from an HTTP-over-SNI placeholder to gRPC, and make gRPC the only transport. Single-purpose: transport only. The old HTTP path never ran against a real server, so dropping it carries no compatibility risk.This is the node/client half of the live-patching path. The proxy/server half (kube-api-proxy Envoy routing to the live-patching service) is already merged separately, so this client is written to match that route.
Flow
flowchart LR A[check-hotfix<br/>node client] -->|1. IMDS GET| B[IMDS attested doc] A -->|2. GetComponentConfig<br/>SNI = apiserver FQDN + ALPN aks-live-patching<br/>token in gRPC metadata| C[kube-api-proxy envoy] C -->|ALPN-routed| D[live-patching service] D -->|config bytes| A A -->|3. parse + stage pointer| E[on-disk hotfix pointer] E -->|read unchanged| F[download-hotfix]Key points
Shared contract, one source of truth. Reuses the
akslivepatching.v1.LivePatchingServicestubs from theaks-live-patchingmodule (the agreed cross-team contract), wired in via a localreplacedirective. No proto or generated code is added in this module.ALPN-based routing (matches the merged proxy route). The client dials the cluster apiserver FQDN as SNI -- riding the existing apiserver egress rule, no special SNI host -- and advertises ALPN
["aks-live-patching", "h2"]. The kube-api-proxy Envoy selects its live-patching filter chain by that ALPN value and forwards the stream to the service. TLS is verified against the on-disk cluster CA.Auth. The IMDS attested-data document rides in gRPC request metadata (the analogue of the old HTTP bearer header).
Unchanged. The parse/stage path, the on-disk pointer file read by
download-hotfix, and thehotfixConfigtype all stay exactly as onmain.check-hotfixremains fail-open (always exits 0).Error mapping (gRPC status -> existing taxonomy):
Unauthenticated/PermissionDenied/NotFoundInvalidArgument/ResourceExhaustedUnavailable/DeadlineExceeded/Internal, transport errorsPlaceholders pending the finalized service contract
ancComponentName(component_namevalue) -- defaults toaks-node-controller.lpsAttestedMetadataKey(metadata key for the attested-data document).Tests
checkhotfix_grpc_test.gouses an in-process bufconn mock of the sharedLivePatchingService: success, benign codes (no-op),Unavailable(fallback),InvalidArgument(no fallback), and fail-open exit 0.go build ./...andgo vetclean. Remaining test failures are pre-existing and environment-specific (need a Linux/etc/os-release+ shell); they fail identically onmainand pass in Linux CI.Draft / POC.