fix: list extension manager resources once per reconcile - #9764
Merged
Conversation
✅ Deploy Preview for cerulean-figolla-1f9435 ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2bc05dfbdb
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
|
||
| // The extension ref filters and backend resources are cluster-wide, | ||
| // so they are collected once per reconcile instead of once per Gateway. | ||
| if err := r.populateExtensionResources(ctx, resourceMap); err != nil { |
There was a problem hiding this comment.
When one controller manages multiple GatewayClasses, Reconcile invokes processGateways separately for every class, so this call still lists every registered extension-filter and custom-backend GVK once per GatewayClass—not once per reconcile as intended. GatewayClasses with no Gateways now also perform these lists, whereas they performed none previously. Fetch the cluster-wide resources once before the managed-GatewayClass loop and populate each class's resource map from that snapshot.
Useful? React with 👍 / 👎.
Contributor
Author
There was a problem hiding this comment.
Hoisting lists above the managed GC loop can't be part of this PR bc break the idiomatic of the surrounding code.
stillya
force-pushed
the
feat/remove-ext-server-hotspot
branch
2 times, most recently
from
August 16, 2026 17:42
a931749 to
6b1dcd3
Compare
Contributor
Author
|
Codex Review: Didn't find any major issues. 🚀 Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
Contributor
Author
Member
|
lgtm, can you add release notes? |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #9764 +/- ##
==========================================
- Coverage 76.36% 76.33% -0.04%
==========================================
Files 261 261
Lines 44537 44543 +6
==========================================
- Hits 34011 34002 -9
- Misses 8279 8293 +14
- Partials 2247 2248 +1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Contributor
Author
Done. |
stillya
force-pushed
the
feat/remove-ext-server-hotspot
branch
3 times, most recently
from
August 17, 2026 08:55
ab76776 to
8dcf409
Compare
zirain
changed the title
zirain
previously approved these changes
Aug 21, 2026
zirain
force-pushed
the
feat/remove-ext-server-hotspot
branch
from
August 21, 2026 11:40
8dcf409 to
7819369
Compare
Member
|
@stillya can you fix the conflict? |
Collect ext-GVK in populateExtensionResources and call it once per reconcile from processGateways. Signed-off-by: stillya <st.ilya.101@gmail.com>
stillya
force-pushed
the
feat/remove-ext-server-hotspot
branch
from
August 25, 2026 08:24
5d9987c to
eaf24a5
Compare
Contributor
Author
Done |
zirain
approved these changes
Aug 25, 2026
kkk777-7
approved these changes
Aug 26, 2026
Member
|
LGTM, thanks! |
kkk777-7
pushed a commit
to kkk777-7/gateway
that referenced
this pull request
Aug 27, 2026
…9764) feat: list extension manager resources once per reconcile Collect ext-GVK in populateExtensionResources and call it once per reconcile from processGateways. Signed-off-by: stillya <st.ilya.101@gmail.com>
zirain
pushed a commit
to zirain/gateway
that referenced
this pull request
Aug 28, 2026
…9764) feat: list extension manager resources once per reconcile Collect ext-GVK in populateExtensionResources and call it once per reconcile from processGateways. Signed-off-by: stillya <st.ilya.101@gmail.com>
zirain
added a commit
that referenced
this pull request
Aug 28, 2026
* Revert 9532 (#9747) * Revert "chore: regen xds translator testdata for initialFetchTimeout (#9643)" This reverts commit 4898608. Signed-off-by: kkk777-7 <kota.kimura0725@gmail.com> * Revert "fix: initial fetch timed out for type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.Secret (#9532)" This reverts commit 4ce3a07. Signed-off-by: kkk777-7 <kota.kimura0725@gmail.com> * regen: testdata output Signed-off-by: kkk777-7 <kota.kimura0725@gmail.com> * fix gen Signed-off-by: zirain <zirain2009@gmail.com> --------- Signed-off-by: kkk777-7 <kota.kimura0725@gmail.com> Signed-off-by: zirain <zirain2009@gmail.com> Co-authored-by: zirain <zirain2009@gmail.com> * fix: rate limiting cannot scale or enforce correctly (#9814) * change RateLimit cluster to EDS type Signed-off-by: zirain <zirain2009@gmail.com> * update test Signed-off-by: zirain <zirain2009@gmail.com> * release notes Signed-off-by: zirain <zirain2009@gmail.com> --------- Signed-off-by: zirain <zirain2009@gmail.com> * fix: udp consistent hash (#9826) * feat: udp consistent hash Signed-off-by: kkk777-7 <kota.kimura0725@gmail.com> * add release note Signed-off-by: kkk777-7 <kota.kimura0725@gmail.com> * fix lint Signed-off-by: kkk777-7 <kota.kimura0725@gmail.com> --------- Signed-off-by: kkk777-7 <kota.kimura0725@gmail.com> * fix: honor access log text format when format.type is unset (#9817) * fix: honor access log text format when format.type is unset ProxyAccessLogFormat.Type is optional: the CEL rule on the type only requires that text or json is set when the type is omitted, so `format: {text: "..."}` is accepted by the API server for any sink. The OpenTelemetry sink honors it, File and ALS did not — both tested `format.Type != nil && *format.Type == Text` and fell through to the JSON branch, where a nil format.JSON produced an empty field list and the translator substituted the default JSON fields. The configured text format was dropped with no error, no status condition and no log line. Resolve the effective format type once, before the sink loop: an explicit type wins, otherwise a text-only format resolves to Text and everything else to JSON. The File and ALS sinks now branch on that. The no-sink default path, which built a JSON access log on /dev/stdout regardless of format, has the same root cause and is fixed with it. OpenTelemetry is deliberately left alone: it can carry text and attributes at the same time and already handles the unset type itself. The existing unit case "nil format type with text only uses text for file sink" from #7720 asserted the JSON fallback its own name argues against; its expectation is updated to the text access log. Fixes #9719 Signed-off-by: Andrey Maltsev <maltsev.andrey@gmail.com> * fix: keep JSON when an unset format type carries both text and json The File and ALS sinks and the no-sink default path have always rendered JSON for format: {text, json} with no type, and the API accepts that input. Inferring Text from a non-nil text alone would silently switch those setups from structured JSON to text and break downstream parsing, so infer Text only when json is absent. Signed-off-by: Andrey Maltsev <maltsev.andrey@gmail.com> --------- Signed-off-by: Andrey Maltsev <maltsev.andrey@gmail.com> * security: enable AES-256-GCM OAuth2 cookie encryption by default (#9831) * security: enable AES-256-GCM OAuth2 cookie encryption by default Set envoy.reloadable_features.oauth2_use_gcm_encryption=true and envoy.reloadable_features.oauth2_legacy_cbc_decrypt_compat=false in the proxy bootstrap's global_config static layer, closing the padding oracle in CVE-2026-47775 for OIDC users by default. Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> Signed-off-by: Huabing (Robin) Zhao <huabing@tetrate.io> * docs: note that Replace bootstraps do not get the OAuth2 GCM guards The guards live in the rendered default bootstrap, which spec.bootstrap type Replace discards. Call that out in the release note so OIDC users with a replacement bootstrap know to set them themselves. Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> Signed-off-by: Huabing (Robin) Zhao <huabing@tetrate.io> --------- Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> Signed-off-by: Huabing (Robin) Zhao <huabing@tetrate.io> * fix: watch namespace changes for listener (#9649) * watch namespace changes Signed-off-by: zirain <zirain2009@gmail.com> * fix lint Signed-off-by: zirain <zirain2009@gmail.com> * fix watch and update Signed-off-by: zirain <zirain2009@gmail.com> * remove log Signed-off-by: zirain <zirain2009@gmail.com> * fix lint Signed-off-by: zirain <zirain2009@gmail.com> * fix e2e Signed-off-by: zirain <zirain2009@gmail.com> * release notes Signed-off-by: zirain <zirain2009@gmail.com> * Distinguish namespace events before forcing translation Signed-off-by: zirain <zirain2009@gmail.com> * fix flaky Signed-off-by: zirain <zirain2009@gmail.com> * remove Force Signed-off-by: zirain <zirain2009@gmail.com> * fix lint Signed-off-by: zirain <zirain2009@gmail.com> * remvoe unless namespace check Signed-off-by: zirain <zirain2009@gmail.com> --------- Signed-off-by: zirain <zirain2009@gmail.com> * fix: list extension manager resources once per reconcile (#9764) feat: list extension manager resources once per reconcile Collect ext-GVK in populateExtensionResources and call it once per reconcile from processGateways. Signed-off-by: stillya <st.ilya.101@gmail.com> * fix(metrics): widen watchable_subscribe_duration_seconds bucket spacing (#9812) * fix(metrics): widen watchable_subscribe_duration_seconds bucket spacing Motivation: watchable_subscribe_duration_seconds used buckets {0.001, 0.01, 0.1, 1, 5, 10}, with a 10x gap between 0.1 and 1 and a 5x gap between 1 and 5. Because histogram_quantile interpolates within a bucket, a control plane whose translations take ~2s puts every observation in (1, 5], so p50/p95/p99 all read as flat somewhere in that range regardless of whether the control plane is healthy or steadily degrading, until it crosses the next boundary. A sub-second control plane has the same problem in (0.1, 1]. Approach: Adopt the bucket spacing already used and reviewed in this repo for the k8s rest client latency histogram (rest_client_request_duration_seconds, internal/metrics/restclient/metrics.go), extended with a trailing 120.0 bucket to keep the slow tail visible: {0.005, 0.025, 0.1, 0.25, 0.5, 1.0, 2.0, 4.0, 8.0, 15.0, 30.0, 60.0, 120.0}. Reusing an already-reviewed spacing avoids picking new numbers from scratch. Bucket boundaries are a breaking change for any dashboard/alert referencing the old le values (0.001, 1, 5 no longer exist), so this adds a release note. The shipped Grafana panel for this metric (charts/gateway-addons-helm/dashboards/envoy-gateway-global.json) queries by le dynamically (legendFormat "{{le}}"), so it keeps working unchanged with the new boundaries. Out of scope: the issue also describes a watchable_depth gauge that's always 0 (reads len() of an unbuffered channel). That part needs a staged plan across the shipped Grafana dashboard (repoint a panel, migrate two label_values queries, then decide deprecate vs. redefine) and is left for a separate change. Two other histograms in this repo (internal/provider/kubernetes/metrics.go's status_update_duration_seconds and internal/infrastructure/kubernetes/metrics.go's resource_apply_duration_seconds / resource_delete_duration_seconds) have the same coarse buckets and were left untouched to keep this change scoped to the exact metric named in the report. Validation: - go build ./internal/message/... ./internal/metrics/... (passed) - go test ./internal/message/... ./internal/metrics/... (passed; no test in the repo pins specific bucket values for this histogram) - golangci-lint run --config=tools/linter/golangci-lint/.golangci.yml ./internal/message/... ./internal/metrics/... (clean) - bash tools/hack/check-release-notes-filenames.sh (passed) - gh run list --repo envoyproxy/gateway --branch main --event push --limit 20: lint, gen-check, coverage-test, and build jobs are green on main; only some e2e/conformance jobs and OSV-Scanner are currently red on main itself, pre-existing and unrelated to this change. Report: #9776 Signed-off-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com> Assisted-by: claude-sonnet-5 (via Claude Code) * docs(release-notes): correct removed histogram boundaries list The note previously listed (0.001, 1, 5) as removed le boundaries, but 1.0 is still present in the new buckets while 0.01 and 10 were also dropped. Update the note to list the actual removed set: 0.001, 0.01, 5, 10. Signed-off-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com> * docs(release-notes): rework wording per reviewer suggestion Adopt the clearer phrasing hsubramanianaks suggested: explicitly call out that _sum/_count are unchanged, use le="..." notation for the removed/retained bucket boundaries, and note that histogram_quantile() queries don't need changes since they aggregate by le dynamically. Signed-off-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com> --------- Signed-off-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com> Co-authored-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com> * fix: skip backendResource watch when CRD is missing instead of crashing controller (#9809) * fix: skip backendResource watch when CRD is missing instead of crashing controller When an extension manager declares backendResources pointing at a custom CRD that is absent (or the controller SA lacks list/watch RBAC), the informer's initial LIST fails, WaitForCacheSync times out, and the controller crash-loops — taking down the entire gateway, not just the routes using the custom backend. This adds a checkCRD call before registering each backendResource watch, following the same pattern already used for ServiceImport, Backend, HTTPRouteFilter, and other optional CRDs. If the CRD is missing, the watch is skipped with a warning log instead of failing cache sync. Fixes #9807 Signed-off-by: Lucas Siedlarczyk <lucas.siedlarczyk@gmail.com> * feat: make control plane trace more readable (#9672) * fix: make control plane trace more readable Signed-off-by: zirain <zirain2009@gmail.com> * update Signed-off-by: zirain <zirain2009@gmail.com> * make sure that span end on panic Signed-off-by: zirain <zirain2009@gmail.com> * record queue wait ms Signed-off-by: zirain <zirain2009@gmail.com> * fix Signed-off-by: zirain <zirain2009@gmail.com> * [observability] Add a dedicated span for EnvoyPatchPolicy JSON patch processing JSON patch processing can dominate the xDS translation time when a cluster has many EnvoyPatchPolicies or expensive patches, but today it is hidden inside the Translator.Translate span, so there is no way to tell how much of a multi-second translation is spent applying patches. Start a Translator.processJSONPatches child span, recording the number of policies and how many patches were applied, targeted a missing resource, or failed. The span is only started when the xDS IR actually carries EnvoyPatchPolicies. Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> Signed-off-by: Huabing (Robin) Zhao <huabing@tetrate.io> Signed-off-by: zirain <zirain2009@gmail.com> * fix context Signed-off-by: zirain <zirain2009@gmail.com> * fix review comments Signed-off-by: zirain <zirain2009@gmail.com> * add trace link Signed-off-by: zirain <zirain2009@gmail.com> * nit Signed-off-by: zirain <zirain2009@gmail.com> * apply huabing's change Signed-off-by: zirain <zirain2009@gmail.com> --------- Signed-off-by: zirain <zirain2009@gmail.com> Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> Signed-off-by: Huabing (Robin) Zhao <huabing@tetrate.io> Co-authored-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> * fix: keep orphaned OIDC PKCE cookies off every request (#9644) * fix: keep orphaned OIDC PKCE cookies off every request Envoy mints a nonce (CSRF) and a PKCE code verifier cookie for every authorization flow it starts, keyed by a random per-flow id so that several logins can be in flight at once. On a successful callback it deletes only the pair belonging to the flow that completed, which is deliberate - sweeping every flow would strip the cookies of the other tabs still mid-login. A flow that is started and abandoned therefore leaves its pair behind until it expires, and at the default path "/" those orphans are sent on every request. Combined with a provider that issues large id and access tokens they overflow the inbound request header limit, at which point the callback itself fails and the browser loops back through the flow, minting yet another pair. Envoy only needs the value of these two cookies when it validates the callback, so scope them to the OIDC redirect path. Orphans then stay off ordinary application requests instead of counting against the header limit on each one. This bounds the problem rather than eliminating it - they are still sent to the callback endpoint, and they still expire on their own within csrfTokenTTL. Also name the code verifier cookie CodeVerifier-<suffix>. It was the only one of the seven OAuth2 cookies left at Envoy's default name, so SecurityPolicies sharing a cookie domain all wrote one shared cookie, and logging out of one policy deleted the in-flight flow cookies of the others. The redirect path is only applied when it satisfies the pattern Envoy enforces on a cookie path, which is stricter than a URL path - "," and ";" for example are legal RFC 3986 sub-delims but are rejected. An unrepresentable path falls back to leaving the cookie path unset, which Envoy defaults to "/", rather than failing xDS validation and dropping the route. Fixes #9632 Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> * test: make the OIDC e2e client path-aware The e2e cookie tracker replayed every stored cookie on every request, keyed by name alone. That made it blind to the cookie path, so the scoping this change relies on could regress without any test noticing, and cookies with the same name at two different paths overwrote each other. Track cookies by name and path, honour the RFC 6265 path-match rules when deciding what to send, and drop cookies the server expires. It is deliberately still not a net/http/cookiejar: the OAuth2 filter marks every cookie "secure" and these tests run over plain HTTP, so a spec-compliant jar would store the cookies and then never send them back. With that in place, assert that an authorization flow which has been started but not completed - the state an abandoned flow leaves behind - keeps its nonce and code verifier cookies off ordinary application requests. Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> --------- Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> * fix(listenerset): preserve routes on hostname conflict winner (#9768) fix(listenerset): preserve routes on conflict winner Signed-off-by: Theis Ferré Hjortkjær <daetfh@egmont.com> * update generate files Signed-off-by: kkk777-7 <kota.kimura0725@gmail.com> * remove: each pr's release notes Signed-off-by: kkk777-7 <kota.kimura0725@gmail.com> --------- Signed-off-by: kkk777-7 <kota.kimura0725@gmail.com> Signed-off-by: zirain <zirain2009@gmail.com> Signed-off-by: Andrey Maltsev <maltsev.andrey@gmail.com> Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> Signed-off-by: Huabing (Robin) Zhao <huabing@tetrate.io> Signed-off-by: stillya <st.ilya.101@gmail.com> Signed-off-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com> Signed-off-by: Lucas Siedlarczyk <lucas.siedlarczyk@gmail.com> Signed-off-by: Theis Ferré Hjortkjær <daetfh@egmont.com> Co-authored-by: zirain <zirain2009@gmail.com> Co-authored-by: Andrey Maltsev <maltsev.andrey@gmail.com> Co-authored-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> Co-authored-by: stillya <st.ilya.101@gmail.com> Co-authored-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com> Co-authored-by: Lucas Siedlarczyk <lucas.siedlarczyk@gmail.com> Co-authored-by: Theis Ferré Hjortkjær <daetfh@egmont.com>
zirain
added a commit
that referenced
this pull request
Aug 28, 2026
* fix: deduplicate CA certificates in ClientTrafficPolicy mTLS (#8909) * fix: deduplicate CA certificates in ClientTrafficPolicy mTLS Signed-off-by: hai.yue <20416005+yuehaii@users.noreply.github.com> * update current release note Signed-off-by: hai.yue <20416005+yuehaii@users.noreply.github.com> * fix lint warnings Signed-off-by: hai.yue <20416005+yuehaii@users.noreply.github.com> * de-duplicate cert during append process Signed-off-by: hai.yue <20416005+yuehaii@users.noreply.github.com> * update ca with different certs Signed-off-by: hai.yue <20416005+yuehaii@users.noreply.github.com> * performance optimize: prevent seen been duplicate rebuild when multiple caCertificateRefs are configured. Signed-off-by: hai.yue <20416005+yuehaii@users.noreply.github.com> * fix incorrect release note Signed-off-by: hai.yue <20416005+yuehaii@users.noreply.github.com> --------- Signed-off-by: hai.yue <20416005+yuehaii@users.noreply.github.com> Signed-off-by: Hai <20416005+yuehaii@users.noreply.github.com> Co-authored-by: zirain <zirain2009@gmail.com> * fix: validate SDS URL (#9030) * fix: add validation for SDS URL Signed-off-by: zirain <zirain2009@gmail.com> * fix Signed-off-by: zirain <zirain2009@gmail.com> * lint Signed-off-by: zirain <zirain2009@gmail.com> * fix test Signed-off-by: zirain <zirain2009@gmail.com> * reject UDS with host Signed-off-by: zirain <zirain2009@gmail.com> * fix lint Signed-off-by: zirain <zirain2009@gmail.com> * fix related path check Signed-off-by: zirain <zirain2009@gmail.com> * release notes Signed-off-by: zirain <zirain2009@gmail.com> * fix Signed-off-by: zirain <zirain2009@gmail.com> --------- Signed-off-by: zirain <zirain2009@gmail.com> * fix: handle EC parameters before TLS private keys (#9338) * Handle EC parameters before TLS private keys Signed-off-by: Jules Dutel <jules.dutel@numberly.com> * fix: bound policy ancestors on the fly to avoid O(N^2) translation latency (#9558) * fix: bound policy ancestors on the fly to avoid O(N^2) translation latency When many resources reference the same target, a policy's status.Ancestors list could grow without bound during translation. SetConditionForPolicyAncestor scans the existing ancestors on every call, so building a status with N ancestors was O(N^2); for a BackendTLSPolicy shared by e.g. 10k EnvoyExtensionPolicies this pushed translation time to ~1h. Cap status.Ancestors at maxPolicyAncestors+1 (17) as entries are added: SetConditionForPolicyAncestor evicts the lowest-priority ancestor once the list exceeds the soft cap, keeping every insert O(1) and the total O(N). The single slot above the CRD limit lets the post-processing TruncatePolicyAncestors still detect overflow, cut to 16, and stamp the Aggregated condition. Note: because eviction happens as ancestors are added, before an ancestor's rank is necessarily final, a policy with more than 16 ancestors that later marks a truncated ancestor Overridden shows only the Overridden condition on it - the redundant Accepted condition is dropped. This affects only truncated statuses. Fixes #9539 Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> * update release note Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> --------- Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> * fix: confirm NotFound against the API server before dropping status writes (#9566) * fix: confirm NotFound against the API server before dropping status writes Under high watch-event churn the status updater's cache-backed Get can return NotFound for a freshly-created object that the informer cache has not caught up to yet. apply() then silently skipped the write (and even counted it as a success), so the object was left with an empty status until a controller restart rebuilt the cache from a fresh LIST. Confirm the NotFound against an uncached APIReader before giving up. The extra uncached read only happens on a cache miss, so it adds no meaningful API-server load in steady state. Addresses #9536 Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> * docs: add release note for #9566 Addresses #9536 Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> --------- Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> * fix: make ListenerSet and GRPCRoute watches optional (#9583) * fix: make ListenerSet and GRPCRoute watches optional (#8991) The Kubernetes provider watches, indexes and Lists ListenerSet and GRPCRoute unconditionally. When either CRD is absent, the manager's cache sync fails and Envoy Gateway crash-loops at startup with: error watching resources: no matches for kind "ListenerSet" in version "gateway.networking.k8s.io/v1" Both kinds are in the Gateway API standard channel, but some managed Kubernetes offerings install only a curated subset of it and don't let users add the missing CRDs. GKE's managed gateway-api-crds addon ships Gateway API v1.5 without listenersets and grpcroutes, because the GKE Gateway controller doesn't implement them; installing them out-of-band means a second owner for addon-managed CRDs. Guard both watches, their indexers, the reconcile-time processing and the GRPCRoute predicate Lists behind a CRD existence check, restoring the pre-#8365 behavior for these two kinds only. TLSRoute and BackendTLSPolicy stay unconditional. When the CRDs are present, behavior is unchanged; when one is absent, the controller starts and simply doesn't serve that kind instead of taking down the whole controller. The offline (file) provider keeps assuming all CRDs are present. Existence is probed once at startup, so a CRD installed later is not watched until the controller restarts, the same trade-off already accepted for EnvoyProxy, TCPRoute and UDPRoute. Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> * fix: make the TLSRoute watch optional too TLSRoute graduated to the Gateway API standard channel in v1.5, so it is subject to the same problem as ListenerSet and GRPCRoute: a cluster may run a bundle that predates the graduation, or a managed offering may ship only a curated subset of the standard channel, and the unconditional watch then fails the manager's cache sync and crash-loops Envoy Gateway at startup. Guard the TLSRoute watch, its indexers, the reconcile-time processing and the TLSRoute predicate List behind a CRD existence check, matching what the previous commit does for ListenerSet and GRPCRoute. Also correct the rationale comment: TCPRoute and UDPRoute are standard channel as of Gateway API v1.6, they are not experimental-only. Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> --------- Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> * fix: don't set Deployment replicas when an HPA is configured (#9594) * fix: don't set Deployment replicas when an HPA is configured Envoy Gateway applies the proxy and rate limit Deployments with a server-side apply patch using ForceOwnership, so rendering spec.replicas made envoy-gateway the owner of that field. Every subsequent reconciliation then forced the replica count back to the statically configured value, overriding whatever the HPA had computed. Omit the replicas field from the rendered Deployment when an HPA is configured, so Envoy Gateway never claims ownership of spec.replicas and the HPA is free to scale the Deployment. This is the behavior already documented for both envoyHpa and rateLimitHpa, but it was never implemented. Use minReplicas to set a lower bound on the replica count instead. Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> * docs: add release note Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> * test: regenerate helm CRD golden files Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> --------- Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> * fix(provider): make BackendTLSPolicy watch optional (#9517) fix(provider): skip absent optional Gateway API CRD watches Envoy Gateway unconditionally establishes watches (and reconcile-time List calls) for ListenerSet, TLSRoute and BackendTLSPolicy. Those types are in the gateway.networking.k8s.io v1 standard channel now, but a cluster can still lack them when it ships an older standard bundle. OpenShift is one example: it installs a fixed standard-channel set and blocks anyone else from adding the rest. When one of these CRDs is missing the manager fails its cache sync and the controller crash-loops with `no matches for kind "..."`. Guard the three watches with a CRD existence check and a skip log, and guard the matching processing and predicate paths, mirroring what is already done for EnvoyProxy, TCPRoute and UDPRoute. The offline provider continues to assume all CRDs exist. When a CRD is present behaviour is unchanged; when it is absent the controller starts and serves the CRDs that do exist. Signed-off-by: Seth Malaki <seth@tigera.io> Co-authored-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> * fix: reconcile ListenerSet when referenced TLS Secret changes (#9634) * fix: reconcile ListenerSet when referenced TLS Secret changes Signed-off-by: zanarelli <zanarelli.dev@gmail.com> * fix: validate parent Gateway for ListenerSet secret watches Mirror isGatewayReferencingSecret by checking that every ListenerSet indexed by a TLS Secret belongs to a Gateway managed by this controller. Signed-off-by: zanarelli <zanarelli.dev@gmail.com> * fix: do not abort secret reconcile on one bad parentRef isGatewayReferencingSecret and isListenerSetReferencingSecret returned false as soon as any unmanaged or missing parent appeared, so a valid Gateway/ListenerSet sharing the same TLS Secret never reconciled. Return true on the first managed parent; continue past Get errors and unmanaged gateways. Cover mixed valid/invalid parents in unit tests. Signed-off-by: zanarelli <zanarelli.dev@gmail.com> --------- Signed-off-by: zanarelli <zanarelli.dev@gmail.com> * fix: pin consistent-hash clients across weighted backendRefs (#9629) When a route splits traffic across multiple weighted backendRefs and uses a ConsistentHash load balancer, Envoy Gateway rendered the split as a weighted_clusters route action whose cluster selection is random per request. The route hash policy only pinned endpoint selection within a cluster, so a client was not pinned to a single backend across the split. Set WeightedCluster.use_hash_policy on the generated weighted clusters when a hash policy is present, so Envoy selects the weighted cluster deterministically from the request's hash policy instead of at random. Gated on a hash policy being configured, so non-ConsistentHash weighted routes are unaffected. Fixes #9626 Signed-off-by: Andrey Maltsev <maltsev.andrey@gmail.com> * fix: watch namespace changes for listener (#9649) * watch namespace changes Signed-off-by: zirain <zirain2009@gmail.com> * fix lint Signed-off-by: zirain <zirain2009@gmail.com> * fix watch and update Signed-off-by: zirain <zirain2009@gmail.com> * remove log Signed-off-by: zirain <zirain2009@gmail.com> * fix lint Signed-off-by: zirain <zirain2009@gmail.com> * fix e2e Signed-off-by: zirain <zirain2009@gmail.com> * release notes Signed-off-by: zirain <zirain2009@gmail.com> * Distinguish namespace events before forcing translation Signed-off-by: zirain <zirain2009@gmail.com> * fix flaky Signed-off-by: zirain <zirain2009@gmail.com> * remove Force Signed-off-by: zirain <zirain2009@gmail.com> * fix lint Signed-off-by: zirain <zirain2009@gmail.com> * remvoe unless namespace check Signed-off-by: zirain <zirain2009@gmail.com> --------- Signed-off-by: zirain <zirain2009@gmail.com> * feat: make control plane trace more readable (#9672) * fix: make control plane trace more readable Signed-off-by: zirain <zirain2009@gmail.com> * update Signed-off-by: zirain <zirain2009@gmail.com> * make sure that span end on panic Signed-off-by: zirain <zirain2009@gmail.com> * record queue wait ms Signed-off-by: zirain <zirain2009@gmail.com> * fix Signed-off-by: zirain <zirain2009@gmail.com> * [observability] Add a dedicated span for EnvoyPatchPolicy JSON patch processing JSON patch processing can dominate the xDS translation time when a cluster has many EnvoyPatchPolicies or expensive patches, but today it is hidden inside the Translator.Translate span, so there is no way to tell how much of a multi-second translation is spent applying patches. Start a Translator.processJSONPatches child span, recording the number of policies and how many patches were applied, targeted a missing resource, or failed. The span is only started when the xDS IR actually carries EnvoyPatchPolicies. Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> Signed-off-by: Huabing (Robin) Zhao <huabing@tetrate.io> Signed-off-by: zirain <zirain2009@gmail.com> * fix context Signed-off-by: zirain <zirain2009@gmail.com> * fix review comments Signed-off-by: zirain <zirain2009@gmail.com> * add trace link Signed-off-by: zirain <zirain2009@gmail.com> * nit Signed-off-by: zirain <zirain2009@gmail.com> * apply huabing's change Signed-off-by: zirain <zirain2009@gmail.com> --------- Signed-off-by: zirain <zirain2009@gmail.com> Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> Signed-off-by: Huabing (Robin) Zhao <huabing@tetrate.io> Co-authored-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> * fix: keep orphaned OIDC PKCE cookies off every request (#9644) * fix: keep orphaned OIDC PKCE cookies off every request Envoy mints a nonce (CSRF) and a PKCE code verifier cookie for every authorization flow it starts, keyed by a random per-flow id so that several logins can be in flight at once. On a successful callback it deletes only the pair belonging to the flow that completed, which is deliberate - sweeping every flow would strip the cookies of the other tabs still mid-login. A flow that is started and abandoned therefore leaves its pair behind until it expires, and at the default path "/" those orphans are sent on every request. Combined with a provider that issues large id and access tokens they overflow the inbound request header limit, at which point the callback itself fails and the browser loops back through the flow, minting yet another pair. Envoy only needs the value of these two cookies when it validates the callback, so scope them to the OIDC redirect path. Orphans then stay off ordinary application requests instead of counting against the header limit on each one. This bounds the problem rather than eliminating it - they are still sent to the callback endpoint, and they still expire on their own within csrfTokenTTL. Also name the code verifier cookie CodeVerifier-<suffix>. It was the only one of the seven OAuth2 cookies left at Envoy's default name, so SecurityPolicies sharing a cookie domain all wrote one shared cookie, and logging out of one policy deleted the in-flight flow cookies of the others. The redirect path is only applied when it satisfies the pattern Envoy enforces on a cookie path, which is stricter than a URL path - "," and ";" for example are legal RFC 3986 sub-delims but are rejected. An unrepresentable path falls back to leaving the cookie path unset, which Envoy defaults to "/", rather than failing xDS validation and dropping the route. Fixes #9632 Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> * test: make the OIDC e2e client path-aware The e2e cookie tracker replayed every stored cookie on every request, keyed by name alone. That made it blind to the cookie path, so the scoping this change relies on could regress without any test noticing, and cookies with the same name at two different paths overwrote each other. Track cookies by name and path, honour the RFC 6265 path-match rules when deciding what to send, and drop cookies the server expires. It is deliberately still not a net/http/cookiejar: the OAuth2 filter marks every cookie "secure" and these tests run over plain HTTP, so a spec-compliant jar would store the cookies and then never send them back. With that in place, assert that an authorization flow which has been started but not completed - the state an abandoned flow leaves behind - keeps its nonce and code verifier cookies off ordinary application requests. Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> --------- Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> Signed-off-by: zirain <zirain2009@gmail.com> * fix: rate limiting cannot scale or enforce correctly (#9814) * change RateLimit cluster to EDS type Signed-off-by: zirain <zirain2009@gmail.com> * update test Signed-off-by: zirain <zirain2009@gmail.com> * release notes Signed-off-by: zirain <zirain2009@gmail.com> --------- Signed-off-by: zirain <zirain2009@gmail.com> * fix: honor access log text format when format.type is unset (#9817) * fix: honor access log text format when format.type is unset ProxyAccessLogFormat.Type is optional: the CEL rule on the type only requires that text or json is set when the type is omitted, so `format: {text: "..."}` is accepted by the API server for any sink. The OpenTelemetry sink honors it, File and ALS did not — both tested `format.Type != nil && *format.Type == Text` and fell through to the JSON branch, where a nil format.JSON produced an empty field list and the translator substituted the default JSON fields. The configured text format was dropped with no error, no status condition and no log line. Resolve the effective format type once, before the sink loop: an explicit type wins, otherwise a text-only format resolves to Text and everything else to JSON. The File and ALS sinks now branch on that. The no-sink default path, which built a JSON access log on /dev/stdout regardless of format, has the same root cause and is fixed with it. OpenTelemetry is deliberately left alone: it can carry text and attributes at the same time and already handles the unset type itself. The existing unit case "nil format type with text only uses text for file sink" from #7720 asserted the JSON fallback its own name argues against; its expectation is updated to the text access log. Fixes #9719 Signed-off-by: Andrey Maltsev <maltsev.andrey@gmail.com> * fix: keep JSON when an unset format type carries both text and json The File and ALS sinks and the no-sink default path have always rendered JSON for format: {text, json} with no type, and the API accepts that input. Inferring Text from a non-nil text alone would silently switch those setups from structured JSON to text and break downstream parsing, so infer Text only when json is absent. Signed-off-by: Andrey Maltsev <maltsev.andrey@gmail.com> --------- Signed-off-by: Andrey Maltsev <maltsev.andrey@gmail.com> * fix: udp consistent hash (#9826) * feat: udp consistent hash Signed-off-by: kkk777-7 <kota.kimura0725@gmail.com> * add release note Signed-off-by: kkk777-7 <kota.kimura0725@gmail.com> * fix lint Signed-off-by: kkk777-7 <kota.kimura0725@gmail.com> --------- Signed-off-by: kkk777-7 <kota.kimura0725@gmail.com> * security: enable AES-256-GCM OAuth2 cookie encryption by default (#9831) * security: enable AES-256-GCM OAuth2 cookie encryption by default Set envoy.reloadable_features.oauth2_use_gcm_encryption=true and envoy.reloadable_features.oauth2_legacy_cbc_decrypt_compat=false in the proxy bootstrap's global_config static layer, closing the padding oracle in CVE-2026-47775 for OIDC users by default. Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> Signed-off-by: Huabing (Robin) Zhao <huabing@tetrate.io> * docs: note that Replace bootstraps do not get the OAuth2 GCM guards The guards live in the rendered default bootstrap, which spec.bootstrap type Replace discards. Call that out in the release note so OIDC users with a replacement bootstrap know to set them themselves. Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> Signed-off-by: Huabing (Robin) Zhao <huabing@tetrate.io> --------- Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> Signed-off-by: Huabing (Robin) Zhao <huabing@tetrate.io> * fix: list extension manager resources once per reconcile (#9764) feat: list extension manager resources once per reconcile Collect ext-GVK in populateExtensionResources and call it once per reconcile from processGateways. Signed-off-by: stillya <st.ilya.101@gmail.com> * fix: skip backendResource watch when CRD is missing instead of crashing controller (#9809) * fix: skip backendResource watch when CRD is missing instead of crashing controller When an extension manager declares backendResources pointing at a custom CRD that is absent (or the controller SA lacks list/watch RBAC), the informer's initial LIST fails, WaitForCacheSync times out, and the controller crash-loops — taking down the entire gateway, not just the routes using the custom backend. This adds a checkCRD call before registering each backendResource watch, following the same pattern already used for ServiceImport, Backend, HTTPRouteFilter, and other optional CRDs. If the CRD is missing, the watch is skipped with a warning log instead of failing cache sync. Fixes #9807 Signed-off-by: Lucas Siedlarczyk <lucas.siedlarczyk@gmail.com> * fix merge Signed-off-by: zirain <zirain2009@gmail.com> * fix Signed-off-by: zirain <zirain2009@gmail.com> --------- Signed-off-by: hai.yue <20416005+yuehaii@users.noreply.github.com> Signed-off-by: Hai <20416005+yuehaii@users.noreply.github.com> Signed-off-by: zirain <zirain2009@gmail.com> Signed-off-by: Jules Dutel <jules.dutel@numberly.com> Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> Signed-off-by: Seth Malaki <seth@tigera.io> Signed-off-by: zanarelli <zanarelli.dev@gmail.com> Signed-off-by: Andrey Maltsev <maltsev.andrey@gmail.com> Signed-off-by: Huabing (Robin) Zhao <huabing@tetrate.io> Signed-off-by: kkk777-7 <kota.kimura0725@gmail.com> Signed-off-by: stillya <st.ilya.101@gmail.com> Signed-off-by: Lucas Siedlarczyk <lucas.siedlarczyk@gmail.com> Co-authored-by: Hai <20416005+yuehaii@users.noreply.github.com> Co-authored-by: jvlxz <jules.dutel@gmail.com> Co-authored-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> Co-authored-by: Seth Malaki <seth.malaki@gmail.com> Co-authored-by: zanarelli <zanarelli.dev@gmail.com> Co-authored-by: Andrey Maltsev <maltsev.andrey@gmail.com> Co-authored-by: Kota Kimura <86363983+kkk777-7@users.noreply.github.com> Co-authored-by: Starchenko Ilya <st.ilya.101@gmail.com> Co-authored-by: Lucas Siedlarczyk <lucas.siedlarczyk@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
Collect ext-GVK in populateExtensionResources and call it once per reconcile from processGateways.
Which issue(s) this PR fixes:
Fixes #9407
PR Checklist
git commit -s). See DCO: Sign your work.make generate gen-check,make lint, and the unit-test/coverage build pass. (Flaky e2e failures are not considered breakages, butgen-check,lint, and coverage MUST pass.)make gen-checkand committed the result if API/helm charts/modules changed.release-notes/current/breaking_changes/.