Skip to content

fix: skip backendResource watch when CRD is missing instead of crashing controller - #9809

Merged
arkodg merged 9 commits into
envoyproxy:mainfrom
Siedlarczyk:fix/backend-resource-crd-graceful-degradation-v2
Aug 27, 2026
Merged

fix: skip backendResource watch when CRD is missing instead of crashing controller#9809
arkodg merged 9 commits into
envoyproxy:mainfrom
Siedlarczyk:fix/backend-resource-crd-graceful-degradation-v2

Conversation

@Siedlarczyk

Copy link
Copy Markdown
Contributor

What this PR does

When an extension manager declares backendResources pointing at a custom CRD that is absent — or the controller's ServiceAccount lacks list/watch RBAC on it — the informer's initial LIST fails, WaitForCacheSync times out, and the controller crash-loops. This takes down the entire gateway controller, not just routes using the custom backend.

This PR adds a checkCRD call before registering each backendResources watch, following the same pattern already used for ServiceImport, Backend, HTTPRouteFilter, and other optional CRDs. If the CRD is missing, the watch is skipped with a warning log instead of propagating the error and failing cache sync.

Before

for _, gvk := range r.extBackendGVKs {
    u := &unstructured.Unstructured{}
    u.SetGroupVersionKind(gvk)
    if err := c.Watch(source.Kind(mgr.GetCache(), u, ...)); err != nil {
        return err  // crashes controller
    }
}

After

for _, gvk := range r.extBackendGVKs {
    crdExists, err := checkCRD(gvk.Kind, gvk.GroupVersion().String())
    if err != nil {
        return err
    }
    if !crdExists {
        r.log.Info("backend resource CRD not found, skipping watch", "resource", gvk.String())
        continue
    }
    // ... register watch as before
}

Validation

Tested in kind with Envoy Gateway v1.8.1 (image rebuilt with this patch):

  • CRD absent: controller starts normally, logs backend resource CRD not found, skipping watch, 0 restarts, no crash-loop.
  • CRD + RBAC installed after restart: controller registers the watch, logs Watching additional backend resource, 0 restarts.

Fixes #9807

@Siedlarczyk
Siedlarczyk requested a review from a team as a code owner August 20, 2026 16:51
@netlify

netlify Bot commented Aug 20, 2026

Copy link
Copy Markdown

Deploy Preview for cerulean-figolla-1f9435 ready!

Name Link
🔨 Latest commit 60d3a99
🔍 Latest deploy log https://app.netlify.com/projects/cerulean-figolla-1f9435/deploys/6a8d279784cab100082c5eb3
😎 Deploy Preview https://deploy-preview-9809--cerulean-figolla-1f9435.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: dac43fb372

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread internal/provider/kubernetes/controller.go
Comment thread internal/provider/kubernetes/controller.go
@Siedlarczyk

Copy link
Copy Markdown
Contributor Author

Also pushed the recommended fix for the filter.go reconciliation process

@zirain

zirain commented Aug 21, 2026

Copy link
Copy Markdown
Member

@codex review

@zirain

zirain commented Aug 21, 2026

Copy link
Copy Markdown
Member

please add a release notes for backport.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 37ccfea15e

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread internal/provider/kubernetes/filters.go Outdated
Comment on lines +46 to +52
if err := r.client.List(ctx, uExtResourceList, client.UnsafeDisableDeepCopy); err != nil {
r.log.Info("no associated backend resources found", "GVK", gvk.String())
return nil, fmt.Errorf("failed to list %s: %w", gvk.String(), err)
// Skip GVKs whose CRD is missing or RBAC is insufficient instead of
// aborting the entire reconcile. The watch for this GVK was already
// skipped in the controller setup; skipping the list path as well
// keeps Gateway processing alive for otherwise valid routes.
r.log.Info("skipping backend resource list", "GVK", gvk.String(), "error", err.Error())
continue

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve errors unrelated to unavailable backend GVKs

When List fails for a transient or unexpected reason (for example a timeout, cache failure, or canceled context), this now treats the GVK as having no objects and lets reconciliation publish an incomplete resource snapshot. Routes using previously available custom backends are consequently translated as BackendNotFound, potentially replacing working configuration during a temporary failure. Only suppress errors that specifically identify the intended missing/unauthorized-resource cases; return other errors so the reconcile is retried.

Useful? React with 👍 / 👎.

@codecov

codecov Bot commented Aug 21, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 50.00000% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 76.34%. Comparing base (4de02ef) to head (60d3a99).
⚠️ Report is 8 commits behind head on main.

Files with missing lines Patch % Lines
internal/provider/kubernetes/controller.go 12.50% 7 Missing ⚠️

❌ Your patch check has failed because the patch coverage (50.00%) is below the target coverage (60.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #9809   +/-   ##
=======================================
  Coverage   76.34%   76.34%           
=======================================
  Files         261      261           
  Lines       44523    44536   +13     
=======================================
+ Hits        33989    33999   +10     
- Misses       8285     8292    +7     
+ Partials     2249     2245    -4     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@Siedlarczyk

Copy link
Copy Markdown
Contributor Author

Hi, @zirain I added some more commits, following the guidance review from codex. I was continuing no matter what the error was, but it made more sense as the review suggested to isolate missing CRDs from other errors (timeouts, etc). Hope it gets clearer now

zirain
zirain previously approved these changes Aug 21, 2026
@Siedlarczyk

Copy link
Copy Markdown
Contributor Author

Fixed some linting issue, should be fine now

@Siedlarczyk
Siedlarczyk requested a review from zirain August 21, 2026 02:30
zirain
zirain previously approved these changes Aug 23, 2026
uExtResourceList := &unstructured.UnstructuredList{}
uExtResourceList.SetGroupVersionKind(gvk)
if err := r.client.List(ctx, uExtResourceList, client.UnsafeDisableDeepCopy); err != nil {
r.log.Info("no associated backend resources found", "GVK", gvk.String())

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

curious what is the impact of keeping the code in the initial state ( plus the above CRD check change)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @arkodg, tks for the comment. I am not sure whether I got you exact doubt (my bad if it's clear, this is my first contribution lol). If you are referring to the filter.go change in its entirety, the idea is pretty much to not break gateways conciliation process due to the absence of a CRD, which might be an error or many other scenarios. As we add gracefully degradation to the controller spin up, it made sense to me the abscence of CRDs don t crash the conciliation of the gateways. And it was also commenting spotted on by the review from codex, which makes sense to me. But if you are referring to treating the absence from other errors, is pretty much to only continue in the absence of CRDs, while other issues return error to the caller function, to retry the operation for instance. Hope that's clear enough, but I am ready anyways. Tks for the review!

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ack, we do have a pattern of saving the crdExists field and using that to skip vs skipping on error types

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can work this, no problem. Just wanted to avoid changing the central structs of the reconciler, and make it simpler. I can add a new field, something like extBackendCRDExists map[schema.GroupVersionKind]bool, and change the code for the spin up to populate the struct. Just wanted to make it simpler and less invasive change. what do you think?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah this works thanks

@Siedlarczyk
Siedlarczyk requested a review from arkodg August 24, 2026 06:39
@Siedlarczyk
Siedlarczyk force-pushed the fix/backend-resource-crd-graceful-degradation-v2 branch from 0dd0535 to c7a6dd3 Compare August 24, 2026 06:45
@Siedlarczyk
Siedlarczyk requested a review from zirain August 24, 2026 06:49
…ng controller

When an extension manager declares backendResources pointing at a custom
CRD that is absent (or the controller SA lacks list/watch RBAC), the
informer's initial LIST fails, WaitForCacheSync times out, and the
controller crash-loops — taking down the entire gateway, not just the
routes using the custom backend.

This adds a checkCRD call before registering each backendResource watch,
following the same pattern already used for ServiceImport, Backend,
HTTPRouteFilter, and other optional CRDs. If the CRD is missing, the
watch is skipped with a warning log instead of failing cache sync.

Fixes envoyproxy#9807

Signed-off-by: Lucas Siedlarczyk <lucas.siedlarczyk@gmail.com>
The checkCRD guard in the watch setup skips registering a watch for a
missing backendResource CRD, but the GVK remains in r.extBackendGVKs.
During reconcile, getExtensionBackendResources iterates over all
configured GVKs and calls r.client.List for each — a missing CRD causes
the List to fail and the error aborts processHTTPRoutes, leaving
otherwise valid Gateways without configuration.

Skip the failing GVK and continue instead of returning an error, so
Gateway processing stays alive for routes that do not depend on the
missing custom backend.

Part of envoyproxy#9807

Signed-off-by: Lucas Siedlarczyk <lucas.siedlarczyk@gmail.com>
Part of envoyproxy#9807

Signed-off-by: Lucas Siedlarczyk <lucas.siedlarczyk@gmail.com>
After skipping the failing GVK with continue instead of returning an
error, the error return value is always nil. Remove it from the
signature, the caller in processHTTPRoutes, and the test cases to
satisfy the unparam linter.

Part of envoyproxy#9807

Signed-off-by: Lucas Siedlarczyk <lucas.siedlarczyk@gmail.com>
Swallowing every List error treated transient failures (timeouts, cache
errors, canceled contexts) as an empty result, so reconciliation published
an incomplete resource snapshot and routes lost working custom backends
during temporary failures.

Only skip GVKs whose CRD is missing or unauthorized (NotFound /
NoMatch), which is the intended graceful-degradation case. Return all
other errors so the reconcile is retried with the full resource set.

Part of envoyproxy#9807

Signed-off-by: Lucas Siedlarczyk <lucas.siedlarczyk@gmail.com>
The entry was added to v1.9.0.yaml, which is already a released and
frozen notes file. Move it to the per-PR fragment format under
current/bug_fixes/ so it is collected into the next patch release
(v1.8.4 / v1.9.1) instead of editing a published file. Also reflect
the refined error handling from the follow-up commits (only NotFound /
NoMatch errors are suppressed; transient failures are returned).

Part of envoyproxy#9807

Signed-off-by: Lucas Siedlarczyk <lucas.siedlarczyk@gmail.com>
Reorder imports to satisfy gci and fix 'cancelation' typo flagged by
codespell.

Part of envoyproxy#9807

Signed-off-by: Lucas Siedlarczyk <lucas.siedlarczyk@gmail.com>
Follow the established codebase pattern (hrfCRDExists, backendCRDExists,
etc.) by caching the CRD check result at controller startup in a field
on gatewayAPIReconciler instead of inspecting error types at reconcile
time.

- Add extBackendCRDExists map[schema.GroupVersionKind]bool field to the
  reconciler struct, populated during watchResources when checkCRD
  returns true.
- In getExtensionBackendResources, skip GVKs not in the cache map
  before calling List, removing the kerrors.IsNotFound/apimeta.IsNoMatch
  error-type inspection from the reconcile hot path.
- Remove unused kerrors and apimeta imports from filters.go.
- Update controller_offline.go to initialize the map with all GVKs set
  to true (offline mode assumes all CRDs exist).
- Update filters_test.go to use the cache map instead of error-type
  interception, add a test case for CRD-absent skip.

Signed-off-by: Lucas Siedlarczyk <lucas.siedlarczyk@gmail.com>
@zirain
zirain force-pushed the fix/backend-resource-crd-graceful-degradation-v2 branch from c7a6dd3 to b6bb027 Compare August 25, 2026 04:17
@zirain

zirain commented Aug 25, 2026

Copy link
Copy Markdown
Member

@Siedlarczyk can you take a look at the test failure?

The extBackendCRDExists field is only populated during watchResources at
controller startup. Code paths that construct a gatewayAPIReconciler
without going through watchResources (tests, offline mode before init)
leave the map nil. Reading a nil map returns the zero-value (false),
causing every backend GVK to be skipped and breaking route processing.

Treat a nil map as 'CRD exists' (pre-PR behavior) so callers that
don't initialize the field continue to work. Only skip a GVK when the
map has been explicitly populated and the GVK is absent.

Signed-off-by: Lucas Siedlarczyk <lucas.siedlarczyk@gmail.com>
@Siedlarczyk

Siedlarczyk commented Aug 25, 2026

Copy link
Copy Markdown
Contributor Author

@Siedlarczyk can you take a look at the test failure?

@zirain done, it was failing because during the test the new struct was not being filled, resulting in a nil map. I just added a nil guard to my changes that fixes that and is a safer approach if this happens elsewhere. Pretty much not an issue as the watch always starts first and start the map

@zirain

zirain commented Aug 25, 2026

Copy link
Copy Markdown
Member

/retest

@kkk777-7

Copy link
Copy Markdown
Member

LGTM, thanks!

@arkodg
arkodg merged commit f330f3a into envoyproxy:main Aug 27, 2026
65 of 69 checks passed
kkk777-7 pushed a commit to kkk777-7/gateway that referenced this pull request Aug 27, 2026
…ng controller (envoyproxy#9809)

* fix: skip backendResource watch when CRD is missing instead of crashing controller

When an extension manager declares backendResources pointing at a custom
CRD that is absent (or the controller SA lacks list/watch RBAC), the
informer's initial LIST fails, WaitForCacheSync times out, and the
controller crash-loops — taking down the entire gateway, not just the
routes using the custom backend.

This adds a checkCRD call before registering each backendResource watch,
following the same pattern already used for ServiceImport, Backend,
HTTPRouteFilter, and other optional CRDs. If the CRD is missing, the
watch is skipped with a warning log instead of failing cache sync.

Fixes envoyproxy#9807

Signed-off-by: Lucas Siedlarczyk <lucas.siedlarczyk@gmail.com>
zirain pushed a commit to zirain/gateway that referenced this pull request Aug 28, 2026
…ng controller (envoyproxy#9809)

* fix: skip backendResource watch when CRD is missing instead of crashing controller

When an extension manager declares backendResources pointing at a custom
CRD that is absent (or the controller SA lacks list/watch RBAC), the
informer's initial LIST fails, WaitForCacheSync times out, and the
controller crash-loops — taking down the entire gateway, not just the
routes using the custom backend.

This adds a checkCRD call before registering each backendResource watch,
following the same pattern already used for ServiceImport, Backend,
HTTPRouteFilter, and other optional CRDs. If the CRD is missing, the
watch is skipped with a warning log instead of failing cache sync.

Fixes envoyproxy#9807

Signed-off-by: Lucas Siedlarczyk <lucas.siedlarczyk@gmail.com>
zirain added a commit that referenced this pull request Aug 28, 2026
* Revert 9532 (#9747)

* Revert "chore: regen xds translator testdata for initialFetchTimeout (#9643)"

This reverts commit 4898608.

Signed-off-by: kkk777-7 <kota.kimura0725@gmail.com>

* Revert "fix: initial fetch timed out for type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.Secret (#9532)"

This reverts commit 4ce3a07.

Signed-off-by: kkk777-7 <kota.kimura0725@gmail.com>

* regen: testdata output

Signed-off-by: kkk777-7 <kota.kimura0725@gmail.com>

* fix gen

Signed-off-by: zirain <zirain2009@gmail.com>

---------

Signed-off-by: kkk777-7 <kota.kimura0725@gmail.com>
Signed-off-by: zirain <zirain2009@gmail.com>
Co-authored-by: zirain <zirain2009@gmail.com>

* fix: rate limiting cannot scale or enforce correctly (#9814)

* change RateLimit cluster to EDS type

Signed-off-by: zirain <zirain2009@gmail.com>

* update test

Signed-off-by: zirain <zirain2009@gmail.com>

* release notes

Signed-off-by: zirain <zirain2009@gmail.com>

---------

Signed-off-by: zirain <zirain2009@gmail.com>

* fix: udp consistent hash (#9826)

* feat: udp consistent hash

Signed-off-by: kkk777-7 <kota.kimura0725@gmail.com>

* add release note

Signed-off-by: kkk777-7 <kota.kimura0725@gmail.com>

* fix lint

Signed-off-by: kkk777-7 <kota.kimura0725@gmail.com>

---------

Signed-off-by: kkk777-7 <kota.kimura0725@gmail.com>

* fix: honor access log text format when format.type is unset (#9817)

* fix: honor access log text format when format.type is unset

ProxyAccessLogFormat.Type is optional: the CEL rule on the type only requires that
text or json is set when the type is omitted, so `format: {text: "..."}` is accepted
by the API server for any sink. The OpenTelemetry sink honors it, File and ALS did
not — both tested `format.Type != nil && *format.Type == Text` and fell through to
the JSON branch, where a nil format.JSON produced an empty field list and the
translator substituted the default JSON fields. The configured text format was
dropped with no error, no status condition and no log line.

Resolve the effective format type once, before the sink loop: an explicit type wins,
otherwise a text-only format resolves to Text and everything else to JSON. The File
and ALS sinks now branch on that. The no-sink default path, which built a JSON access
log on /dev/stdout regardless of format, has the same root cause and is fixed with it.

OpenTelemetry is deliberately left alone: it can carry text and attributes at the same
time and already handles the unset type itself.

The existing unit case "nil format type with text only uses text for file sink" from
#7720 asserted the JSON fallback its own name argues against; its expectation is
updated to the text access log.

Fixes #9719

Signed-off-by: Andrey Maltsev <maltsev.andrey@gmail.com>

* fix: keep JSON when an unset format type carries both text and json

The File and ALS sinks and the no-sink default path have always rendered JSON for
format: {text, json} with no type, and the API accepts that input. Inferring Text
from a non-nil text alone would silently switch those setups from structured JSON
to text and break downstream parsing, so infer Text only when json is absent.

Signed-off-by: Andrey Maltsev <maltsev.andrey@gmail.com>

---------

Signed-off-by: Andrey Maltsev <maltsev.andrey@gmail.com>

* security: enable AES-256-GCM OAuth2 cookie encryption by default (#9831)

* security: enable AES-256-GCM OAuth2 cookie encryption by default

Set envoy.reloadable_features.oauth2_use_gcm_encryption=true and
envoy.reloadable_features.oauth2_legacy_cbc_decrypt_compat=false in the
proxy bootstrap's global_config static layer, closing the padding oracle
in CVE-2026-47775 for OIDC users by default.

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>
Signed-off-by: Huabing (Robin) Zhao <huabing@tetrate.io>

* docs: note that Replace bootstraps do not get the OAuth2 GCM guards

The guards live in the rendered default bootstrap, which spec.bootstrap
type Replace discards. Call that out in the release note so OIDC users
with a replacement bootstrap know to set them themselves.

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>
Signed-off-by: Huabing (Robin) Zhao <huabing@tetrate.io>

---------

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>
Signed-off-by: Huabing (Robin) Zhao <huabing@tetrate.io>

* fix: watch namespace changes for listener (#9649)

* watch namespace changes

Signed-off-by: zirain <zirain2009@gmail.com>

* fix lint

Signed-off-by: zirain <zirain2009@gmail.com>

* fix watch and update

Signed-off-by: zirain <zirain2009@gmail.com>

* remove log

Signed-off-by: zirain <zirain2009@gmail.com>

* fix lint

Signed-off-by: zirain <zirain2009@gmail.com>

* fix e2e

Signed-off-by: zirain <zirain2009@gmail.com>

* release notes

Signed-off-by: zirain <zirain2009@gmail.com>

* Distinguish namespace events before forcing translation

Signed-off-by: zirain <zirain2009@gmail.com>

* fix flaky

Signed-off-by: zirain <zirain2009@gmail.com>

* remove Force

Signed-off-by: zirain <zirain2009@gmail.com>

* fix lint

Signed-off-by: zirain <zirain2009@gmail.com>

* remvoe unless namespace check

Signed-off-by: zirain <zirain2009@gmail.com>

---------

Signed-off-by: zirain <zirain2009@gmail.com>

* fix: list extension manager resources once per reconcile (#9764)

feat: list extension manager resources once per reconcile
Collect ext-GVK in populateExtensionResources and call it once per reconcile from processGateways.

Signed-off-by: stillya <st.ilya.101@gmail.com>

* fix(metrics): widen watchable_subscribe_duration_seconds bucket spacing (#9812)

* fix(metrics): widen watchable_subscribe_duration_seconds bucket spacing

Motivation:
watchable_subscribe_duration_seconds used buckets {0.001, 0.01, 0.1, 1,
5, 10}, with a 10x gap between 0.1 and 1 and a 5x gap between 1 and 5.
Because histogram_quantile interpolates within a bucket, a control
plane whose translations take ~2s puts every observation in (1, 5],
so p50/p95/p99 all read as flat somewhere in that range regardless of
whether the control plane is healthy or steadily degrading, until it
crosses the next boundary. A sub-second control plane has the same
problem in (0.1, 1].

Approach:
Adopt the bucket spacing already used and reviewed in this repo for
the k8s rest client latency histogram (rest_client_request_duration_seconds,
internal/metrics/restclient/metrics.go), extended with a trailing
120.0 bucket to keep the slow tail visible: {0.005, 0.025, 0.1, 0.25,
0.5, 1.0, 2.0, 4.0, 8.0, 15.0, 30.0, 60.0, 120.0}. Reusing an
already-reviewed spacing avoids picking new numbers from scratch.

Bucket boundaries are a breaking change for any dashboard/alert
referencing the old le values (0.001, 1, 5 no longer exist), so this
adds a release note. The shipped Grafana panel for this metric
(charts/gateway-addons-helm/dashboards/envoy-gateway-global.json)
queries by le dynamically (legendFormat "{{le}}"), so it keeps working
unchanged with the new boundaries.

Out of scope: the issue also describes a watchable_depth gauge that's
always 0 (reads len() of an unbuffered channel). That part needs a
staged plan across the shipped Grafana dashboard (repoint a panel,
migrate two label_values queries, then decide deprecate vs. redefine)
and is left for a separate change. Two other histograms in this repo
(internal/provider/kubernetes/metrics.go's status_update_duration_seconds
and internal/infrastructure/kubernetes/metrics.go's
resource_apply_duration_seconds / resource_delete_duration_seconds)
have the same coarse buckets and were left untouched to keep this
change scoped to the exact metric named in the report.

Validation:
- go build ./internal/message/... ./internal/metrics/... (passed)
- go test ./internal/message/... ./internal/metrics/... (passed; no
  test in the repo pins specific bucket values for this histogram)
- golangci-lint run --config=tools/linter/golangci-lint/.golangci.yml
  ./internal/message/... ./internal/metrics/... (clean)
- bash tools/hack/check-release-notes-filenames.sh (passed)
- gh run list --repo envoyproxy/gateway --branch main --event push
  --limit 20: lint, gen-check, coverage-test, and build jobs are green
  on main; only some e2e/conformance jobs and OSV-Scanner are
  currently red on main itself, pre-existing and unrelated to this
  change.

Report: #9776
Signed-off-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com>
Assisted-by: claude-sonnet-5 (via Claude Code)

* docs(release-notes): correct removed histogram boundaries list

The note previously listed (0.001, 1, 5) as removed le boundaries, but
1.0 is still present in the new buckets while 0.01 and 10 were also
dropped. Update the note to list the actual removed set: 0.001, 0.01,
5, 10.

Signed-off-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com>

* docs(release-notes): rework wording per reviewer suggestion

Adopt the clearer phrasing hsubramanianaks suggested: explicitly call
out that _sum/_count are unchanged, use le="..." notation for the
removed/retained bucket boundaries, and note that histogram_quantile()
queries don't need changes since they aggregate by le dynamically.

Signed-off-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com>

---------

Signed-off-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com>
Co-authored-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com>

* fix: skip backendResource watch when CRD is missing instead of crashing controller (#9809)

* fix: skip backendResource watch when CRD is missing instead of crashing controller

When an extension manager declares backendResources pointing at a custom
CRD that is absent (or the controller SA lacks list/watch RBAC), the
informer's initial LIST fails, WaitForCacheSync times out, and the
controller crash-loops — taking down the entire gateway, not just the
routes using the custom backend.

This adds a checkCRD call before registering each backendResource watch,
following the same pattern already used for ServiceImport, Backend,
HTTPRouteFilter, and other optional CRDs. If the CRD is missing, the
watch is skipped with a warning log instead of failing cache sync.

Fixes #9807

Signed-off-by: Lucas Siedlarczyk <lucas.siedlarczyk@gmail.com>

* feat: make control plane trace more readable (#9672)

* fix: make control plane trace more readable

Signed-off-by: zirain <zirain2009@gmail.com>

* update

Signed-off-by: zirain <zirain2009@gmail.com>

* make sure that span end on panic

Signed-off-by: zirain <zirain2009@gmail.com>

* record queue wait ms

Signed-off-by: zirain <zirain2009@gmail.com>

* fix

Signed-off-by: zirain <zirain2009@gmail.com>

* [observability] Add a dedicated span for EnvoyPatchPolicy JSON patch processing

JSON patch processing can dominate the xDS translation time when a cluster
has many EnvoyPatchPolicies or expensive patches, but today it is hidden
inside the Translator.Translate span, so there is no way to tell how much of
a multi-second translation is spent applying patches.

Start a Translator.processJSONPatches child span, recording the number of
policies and how many patches were applied, targeted a missing resource, or
failed. The span is only started when the xDS IR actually carries
EnvoyPatchPolicies.

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>
Signed-off-by: Huabing (Robin) Zhao <huabing@tetrate.io>
Signed-off-by: zirain <zirain2009@gmail.com>

* fix context

Signed-off-by: zirain <zirain2009@gmail.com>

* fix review comments

Signed-off-by: zirain <zirain2009@gmail.com>

* add trace link

Signed-off-by: zirain <zirain2009@gmail.com>

* nit

Signed-off-by: zirain <zirain2009@gmail.com>

* apply huabing's change

Signed-off-by: zirain <zirain2009@gmail.com>

---------

Signed-off-by: zirain <zirain2009@gmail.com>
Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>
Signed-off-by: Huabing (Robin) Zhao <huabing@tetrate.io>
Co-authored-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* fix: keep orphaned OIDC PKCE cookies off every request (#9644)

* fix: keep orphaned OIDC PKCE cookies off every request

Envoy mints a nonce (CSRF) and a PKCE code verifier cookie for every
authorization flow it starts, keyed by a random per-flow id so that several
logins can be in flight at once. On a successful callback it deletes only the
pair belonging to the flow that completed, which is deliberate - sweeping every
flow would strip the cookies of the other tabs still mid-login. A flow that is
started and abandoned therefore leaves its pair behind until it expires, and at
the default path "/" those orphans are sent on every request. Combined with a
provider that issues large id and access tokens they overflow the inbound
request header limit, at which point the callback itself fails and the browser
loops back through the flow, minting yet another pair.

Envoy only needs the value of these two cookies when it validates the
callback, so scope them to the OIDC redirect path. Orphans then stay off
ordinary application requests instead of counting against the header limit on
each one. This bounds the problem rather than eliminating it - they are still
sent to the callback endpoint, and they still expire on their own within
csrfTokenTTL.

Also name the code verifier cookie CodeVerifier-<suffix>. It was the only one
of the seven OAuth2 cookies left at Envoy's default name, so SecurityPolicies
sharing a cookie domain all wrote one shared cookie, and logging out of one
policy deleted the in-flight flow cookies of the others.

The redirect path is only applied when it satisfies the pattern Envoy enforces
on a cookie path, which is stricter than a URL path - "," and ";" for example
are legal RFC 3986 sub-delims but are rejected. An unrepresentable path falls
back to leaving the cookie path unset, which Envoy defaults to "/", rather
than failing xDS validation and dropping the route.

Fixes #9632

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* test: make the OIDC e2e client path-aware

The e2e cookie tracker replayed every stored cookie on every request, keyed by
name alone. That made it blind to the cookie path, so the scoping this change
relies on could regress without any test noticing, and cookies with the same
name at two different paths overwrote each other.

Track cookies by name and path, honour the RFC 6265 path-match rules when
deciding what to send, and drop cookies the server expires. It is deliberately
still not a net/http/cookiejar: the OAuth2 filter marks every cookie "secure"
and these tests run over plain HTTP, so a spec-compliant jar would store the
cookies and then never send them back.

With that in place, assert that an authorization flow which has been started
but not completed - the state an abandoned flow leaves behind - keeps its nonce
and code verifier cookies off ordinary application requests.

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

---------

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* fix(listenerset): preserve routes on hostname conflict winner (#9768)

fix(listenerset): preserve routes on conflict winner

Signed-off-by: Theis Ferré Hjortkjær <daetfh@egmont.com>

* update generate files

Signed-off-by: kkk777-7 <kota.kimura0725@gmail.com>

* remove: each pr's release notes

Signed-off-by: kkk777-7 <kota.kimura0725@gmail.com>

---------

Signed-off-by: kkk777-7 <kota.kimura0725@gmail.com>
Signed-off-by: zirain <zirain2009@gmail.com>
Signed-off-by: Andrey Maltsev <maltsev.andrey@gmail.com>
Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>
Signed-off-by: Huabing (Robin) Zhao <huabing@tetrate.io>
Signed-off-by: stillya <st.ilya.101@gmail.com>
Signed-off-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com>
Signed-off-by: Lucas Siedlarczyk <lucas.siedlarczyk@gmail.com>
Signed-off-by: Theis Ferré Hjortkjær <daetfh@egmont.com>
Co-authored-by: zirain <zirain2009@gmail.com>
Co-authored-by: Andrey Maltsev <maltsev.andrey@gmail.com>
Co-authored-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>
Co-authored-by: stillya <st.ilya.101@gmail.com>
Co-authored-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com>
Co-authored-by: Lucas Siedlarczyk <lucas.siedlarczyk@gmail.com>
Co-authored-by: Theis Ferré Hjortkjær <daetfh@egmont.com>
zirain added a commit that referenced this pull request Aug 28, 2026
* fix: deduplicate CA certificates in ClientTrafficPolicy mTLS (#8909)

* fix: deduplicate CA certificates in ClientTrafficPolicy mTLS

Signed-off-by: hai.yue <20416005+yuehaii@users.noreply.github.com>

* update current release note

Signed-off-by: hai.yue <20416005+yuehaii@users.noreply.github.com>

* fix lint warnings

Signed-off-by: hai.yue <20416005+yuehaii@users.noreply.github.com>

* de-duplicate cert during append process

Signed-off-by: hai.yue <20416005+yuehaii@users.noreply.github.com>

* update ca with different certs

Signed-off-by: hai.yue <20416005+yuehaii@users.noreply.github.com>

* performance optimize: prevent seen been duplicate rebuild when multiple caCertificateRefs are configured.

Signed-off-by: hai.yue <20416005+yuehaii@users.noreply.github.com>

* fix incorrect release note

Signed-off-by: hai.yue <20416005+yuehaii@users.noreply.github.com>

---------

Signed-off-by: hai.yue <20416005+yuehaii@users.noreply.github.com>
Signed-off-by: Hai <20416005+yuehaii@users.noreply.github.com>
Co-authored-by: zirain <zirain2009@gmail.com>

* fix: validate SDS URL (#9030)

* fix: add validation for SDS URL

Signed-off-by: zirain <zirain2009@gmail.com>

* fix

Signed-off-by: zirain <zirain2009@gmail.com>

* lint

Signed-off-by: zirain <zirain2009@gmail.com>

* fix test

Signed-off-by: zirain <zirain2009@gmail.com>

* reject UDS with host

Signed-off-by: zirain <zirain2009@gmail.com>

* fix lint

Signed-off-by: zirain <zirain2009@gmail.com>

* fix related path check

Signed-off-by: zirain <zirain2009@gmail.com>

* release notes

Signed-off-by: zirain <zirain2009@gmail.com>

* fix

Signed-off-by: zirain <zirain2009@gmail.com>

---------

Signed-off-by: zirain <zirain2009@gmail.com>

* fix: handle EC parameters before TLS private keys (#9338)

* Handle EC parameters before TLS private keys

Signed-off-by: Jules Dutel <jules.dutel@numberly.com>

* fix: bound policy ancestors on the fly to avoid O(N^2) translation latency (#9558)

* fix: bound policy ancestors on the fly to avoid O(N^2) translation latency

When many resources reference the same target, a policy's status.Ancestors list
could grow without bound during translation. SetConditionForPolicyAncestor scans
the existing ancestors on every call, so building a status with N ancestors was
O(N^2); for a BackendTLSPolicy shared by e.g. 10k EnvoyExtensionPolicies this
pushed translation time to ~1h.

Cap status.Ancestors at maxPolicyAncestors+1 (17) as entries are added:
SetConditionForPolicyAncestor evicts the lowest-priority ancestor once the list
exceeds the soft cap, keeping every insert O(1) and the total O(N). The single
slot above the CRD limit lets the post-processing TruncatePolicyAncestors still
detect overflow, cut to 16, and stamp the Aggregated condition.

Note: because eviction happens as ancestors are added, before an ancestor's rank
is necessarily final, a policy with more than 16 ancestors that later marks a
truncated ancestor Overridden shows only the Overridden condition on it - the
redundant Accepted condition is dropped. This affects only truncated statuses.

Fixes #9539

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* update release note

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

---------

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* fix: confirm NotFound against the API server before dropping status writes (#9566)

* fix: confirm NotFound against the API server before dropping status writes

Under high watch-event churn the status updater's cache-backed Get can
return NotFound for a freshly-created object that the informer cache has
not caught up to yet. apply() then silently skipped the write (and even
counted it as a success), so the object was left with an empty status
until a controller restart rebuilt the cache from a fresh LIST.

Confirm the NotFound against an uncached APIReader before giving up. The
extra uncached read only happens on a cache miss, so it adds no meaningful
API-server load in steady state.

Addresses #9536

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* docs: add release note for #9566

Addresses #9536

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

---------

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* fix: make ListenerSet and GRPCRoute watches optional (#9583)

* fix: make ListenerSet and GRPCRoute watches optional (#8991)

The Kubernetes provider watches, indexes and Lists ListenerSet and
GRPCRoute unconditionally. When either CRD is absent, the manager's cache
sync fails and Envoy Gateway crash-loops at startup with:

  error watching resources: no matches for kind "ListenerSet" in version
  "gateway.networking.k8s.io/v1"

Both kinds are in the Gateway API standard channel, but some managed
Kubernetes offerings install only a curated subset of it and don't let
users add the missing CRDs. GKE's managed gateway-api-crds addon ships
Gateway API v1.5 without listenersets and grpcroutes, because the GKE
Gateway controller doesn't implement them; installing them out-of-band
means a second owner for addon-managed CRDs.

Guard both watches, their indexers, the reconcile-time processing and the
GRPCRoute predicate Lists behind a CRD existence check, restoring the
pre-#8365 behavior for these two kinds only. TLSRoute and
BackendTLSPolicy stay unconditional. When the CRDs are present, behavior
is unchanged; when one is absent, the controller starts and simply
doesn't serve that kind instead of taking down the whole controller. The
offline (file) provider keeps assuming all CRDs are present.

Existence is probed once at startup, so a CRD installed later is not
watched until the controller restarts, the same trade-off already
accepted for EnvoyProxy, TCPRoute and UDPRoute.

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* fix: make the TLSRoute watch optional too

TLSRoute graduated to the Gateway API standard channel in v1.5, so it is
subject to the same problem as ListenerSet and GRPCRoute: a cluster may
run a bundle that predates the graduation, or a managed offering may ship
only a curated subset of the standard channel, and the unconditional
watch then fails the manager's cache sync and crash-loops Envoy Gateway
at startup.

Guard the TLSRoute watch, its indexers, the reconcile-time processing and
the TLSRoute predicate List behind a CRD existence check, matching what
the previous commit does for ListenerSet and GRPCRoute. Also correct the
rationale comment: TCPRoute and UDPRoute are standard channel as of
Gateway API v1.6, they are not experimental-only.

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

---------

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* fix: don't set Deployment replicas when an HPA is configured (#9594)

* fix: don't set Deployment replicas when an HPA is configured

Envoy Gateway applies the proxy and rate limit Deployments with a
server-side apply patch using ForceOwnership, so rendering spec.replicas
made envoy-gateway the owner of that field. Every subsequent
reconciliation then forced the replica count back to the statically
configured value, overriding whatever the HPA had computed.

Omit the replicas field from the rendered Deployment when an HPA is
configured, so Envoy Gateway never claims ownership of spec.replicas and
the HPA is free to scale the Deployment. This is the behavior already
documented for both envoyHpa and rateLimitHpa, but it was never
implemented. Use minReplicas to set a lower bound on the replica count
instead.

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* docs: add release note

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* test: regenerate helm CRD golden files

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

---------

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* fix(provider): make BackendTLSPolicy watch optional (#9517)

fix(provider): skip absent optional Gateway API CRD watches

Envoy Gateway unconditionally establishes watches (and reconcile-time
List calls) for ListenerSet, TLSRoute and BackendTLSPolicy. Those types
are in the gateway.networking.k8s.io v1 standard channel now, but a
cluster can still lack them when it ships an older standard bundle.
OpenShift is one example: it installs a fixed standard-channel set and
blocks anyone else from adding the rest. When one of these CRDs is
missing the manager fails its cache sync and the controller crash-loops
with `no matches for kind "..."`.

Guard the three watches with a CRD existence check and a skip log, and
guard the matching processing and predicate paths, mirroring what is
already done for EnvoyProxy, TCPRoute and UDPRoute. The offline provider
continues to assume all CRDs exist. When a CRD is present behaviour is
unchanged; when it is absent the controller starts and serves the CRDs
that do exist.

Signed-off-by: Seth Malaki <seth@tigera.io>
Co-authored-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* fix: reconcile ListenerSet when referenced TLS Secret changes (#9634)

* fix: reconcile ListenerSet when referenced TLS Secret changes

Signed-off-by: zanarelli <zanarelli.dev@gmail.com>

* fix: validate parent Gateway for ListenerSet secret watches

Mirror isGatewayReferencingSecret by checking that every ListenerSet indexed by a TLS Secret belongs to a Gateway managed by this controller.

Signed-off-by: zanarelli <zanarelli.dev@gmail.com>

* fix: do not abort secret reconcile on one bad parentRef

isGatewayReferencingSecret and isListenerSetReferencingSecret returned
false as soon as any unmanaged or missing parent appeared, so a valid
Gateway/ListenerSet sharing the same TLS Secret never reconciled.

Return true on the first managed parent; continue past Get errors and
unmanaged gateways. Cover mixed valid/invalid parents in unit tests.

Signed-off-by: zanarelli <zanarelli.dev@gmail.com>

---------

Signed-off-by: zanarelli <zanarelli.dev@gmail.com>

* fix: pin consistent-hash clients across weighted backendRefs (#9629)

When a route splits traffic across multiple weighted backendRefs and uses a ConsistentHash
load balancer, Envoy Gateway rendered the split as a weighted_clusters route action whose
cluster selection is random per request. The route hash policy only pinned endpoint selection
within a cluster, so a client was not pinned to a single backend across the split.

Set WeightedCluster.use_hash_policy on the generated weighted clusters when a hash policy is
present, so Envoy selects the weighted cluster deterministically from the request's hash policy
instead of at random. Gated on a hash policy being configured, so non-ConsistentHash weighted
routes are unaffected.

Fixes #9626

Signed-off-by: Andrey Maltsev <maltsev.andrey@gmail.com>

* fix: watch namespace changes for listener (#9649)

* watch namespace changes

Signed-off-by: zirain <zirain2009@gmail.com>

* fix lint

Signed-off-by: zirain <zirain2009@gmail.com>

* fix watch and update

Signed-off-by: zirain <zirain2009@gmail.com>

* remove log

Signed-off-by: zirain <zirain2009@gmail.com>

* fix lint

Signed-off-by: zirain <zirain2009@gmail.com>

* fix e2e

Signed-off-by: zirain <zirain2009@gmail.com>

* release notes

Signed-off-by: zirain <zirain2009@gmail.com>

* Distinguish namespace events before forcing translation

Signed-off-by: zirain <zirain2009@gmail.com>

* fix flaky

Signed-off-by: zirain <zirain2009@gmail.com>

* remove Force

Signed-off-by: zirain <zirain2009@gmail.com>

* fix lint

Signed-off-by: zirain <zirain2009@gmail.com>

* remvoe unless namespace check

Signed-off-by: zirain <zirain2009@gmail.com>

---------

Signed-off-by: zirain <zirain2009@gmail.com>

* feat: make control plane trace more readable (#9672)

* fix: make control plane trace more readable

Signed-off-by: zirain <zirain2009@gmail.com>

* update

Signed-off-by: zirain <zirain2009@gmail.com>

* make sure that span end on panic

Signed-off-by: zirain <zirain2009@gmail.com>

* record queue wait ms

Signed-off-by: zirain <zirain2009@gmail.com>

* fix

Signed-off-by: zirain <zirain2009@gmail.com>

* [observability] Add a dedicated span for EnvoyPatchPolicy JSON patch processing

JSON patch processing can dominate the xDS translation time when a cluster
has many EnvoyPatchPolicies or expensive patches, but today it is hidden
inside the Translator.Translate span, so there is no way to tell how much of
a multi-second translation is spent applying patches.

Start a Translator.processJSONPatches child span, recording the number of
policies and how many patches were applied, targeted a missing resource, or
failed. The span is only started when the xDS IR actually carries
EnvoyPatchPolicies.

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>
Signed-off-by: Huabing (Robin) Zhao <huabing@tetrate.io>
Signed-off-by: zirain <zirain2009@gmail.com>

* fix context

Signed-off-by: zirain <zirain2009@gmail.com>

* fix review comments

Signed-off-by: zirain <zirain2009@gmail.com>

* add trace link

Signed-off-by: zirain <zirain2009@gmail.com>

* nit

Signed-off-by: zirain <zirain2009@gmail.com>

* apply huabing's change

Signed-off-by: zirain <zirain2009@gmail.com>

---------

Signed-off-by: zirain <zirain2009@gmail.com>
Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>
Signed-off-by: Huabing (Robin) Zhao <huabing@tetrate.io>
Co-authored-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* fix: keep orphaned OIDC PKCE cookies off every request (#9644)

* fix: keep orphaned OIDC PKCE cookies off every request

Envoy mints a nonce (CSRF) and a PKCE code verifier cookie for every
authorization flow it starts, keyed by a random per-flow id so that several
logins can be in flight at once. On a successful callback it deletes only the
pair belonging to the flow that completed, which is deliberate - sweeping every
flow would strip the cookies of the other tabs still mid-login. A flow that is
started and abandoned therefore leaves its pair behind until it expires, and at
the default path "/" those orphans are sent on every request. Combined with a
provider that issues large id and access tokens they overflow the inbound
request header limit, at which point the callback itself fails and the browser
loops back through the flow, minting yet another pair.

Envoy only needs the value of these two cookies when it validates the
callback, so scope them to the OIDC redirect path. Orphans then stay off
ordinary application requests instead of counting against the header limit on
each one. This bounds the problem rather than eliminating it - they are still
sent to the callback endpoint, and they still expire on their own within
csrfTokenTTL.

Also name the code verifier cookie CodeVerifier-<suffix>. It was the only one
of the seven OAuth2 cookies left at Envoy's default name, so SecurityPolicies
sharing a cookie domain all wrote one shared cookie, and logging out of one
policy deleted the in-flight flow cookies of the others.

The redirect path is only applied when it satisfies the pattern Envoy enforces
on a cookie path, which is stricter than a URL path - "," and ";" for example
are legal RFC 3986 sub-delims but are rejected. An unrepresentable path falls
back to leaving the cookie path unset, which Envoy defaults to "/", rather
than failing xDS validation and dropping the route.

Fixes #9632

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* test: make the OIDC e2e client path-aware

The e2e cookie tracker replayed every stored cookie on every request, keyed by
name alone. That made it blind to the cookie path, so the scoping this change
relies on could regress without any test noticing, and cookies with the same
name at two different paths overwrote each other.

Track cookies by name and path, honour the RFC 6265 path-match rules when
deciding what to send, and drop cookies the server expires. It is deliberately
still not a net/http/cookiejar: the OAuth2 filter marks every cookie "secure"
and these tests run over plain HTTP, so a spec-compliant jar would store the
cookies and then never send them back.

With that in place, assert that an authorization flow which has been started
but not completed - the state an abandoned flow leaves behind - keeps its nonce
and code verifier cookies off ordinary application requests.

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

---------

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>
Signed-off-by: zirain <zirain2009@gmail.com>

* fix: rate limiting cannot scale or enforce correctly (#9814)

* change RateLimit cluster to EDS type

Signed-off-by: zirain <zirain2009@gmail.com>

* update test

Signed-off-by: zirain <zirain2009@gmail.com>

* release notes

Signed-off-by: zirain <zirain2009@gmail.com>

---------

Signed-off-by: zirain <zirain2009@gmail.com>

* fix: honor access log text format when format.type is unset (#9817)

* fix: honor access log text format when format.type is unset

ProxyAccessLogFormat.Type is optional: the CEL rule on the type only requires that
text or json is set when the type is omitted, so `format: {text: "..."}` is accepted
by the API server for any sink. The OpenTelemetry sink honors it, File and ALS did
not — both tested `format.Type != nil && *format.Type == Text` and fell through to
the JSON branch, where a nil format.JSON produced an empty field list and the
translator substituted the default JSON fields. The configured text format was
dropped with no error, no status condition and no log line.

Resolve the effective format type once, before the sink loop: an explicit type wins,
otherwise a text-only format resolves to Text and everything else to JSON. The File
and ALS sinks now branch on that. The no-sink default path, which built a JSON access
log on /dev/stdout regardless of format, has the same root cause and is fixed with it.

OpenTelemetry is deliberately left alone: it can carry text and attributes at the same
time and already handles the unset type itself.

The existing unit case "nil format type with text only uses text for file sink" from
#7720 asserted the JSON fallback its own name argues against; its expectation is
updated to the text access log.

Fixes #9719

Signed-off-by: Andrey Maltsev <maltsev.andrey@gmail.com>

* fix: keep JSON when an unset format type carries both text and json

The File and ALS sinks and the no-sink default path have always rendered JSON for
format: {text, json} with no type, and the API accepts that input. Inferring Text
from a non-nil text alone would silently switch those setups from structured JSON
to text and break downstream parsing, so infer Text only when json is absent.

Signed-off-by: Andrey Maltsev <maltsev.andrey@gmail.com>

---------

Signed-off-by: Andrey Maltsev <maltsev.andrey@gmail.com>

* fix: udp consistent hash (#9826)

* feat: udp consistent hash

Signed-off-by: kkk777-7 <kota.kimura0725@gmail.com>

* add release note

Signed-off-by: kkk777-7 <kota.kimura0725@gmail.com>

* fix lint

Signed-off-by: kkk777-7 <kota.kimura0725@gmail.com>

---------

Signed-off-by: kkk777-7 <kota.kimura0725@gmail.com>

* security: enable AES-256-GCM OAuth2 cookie encryption by default (#9831)

* security: enable AES-256-GCM OAuth2 cookie encryption by default

Set envoy.reloadable_features.oauth2_use_gcm_encryption=true and
envoy.reloadable_features.oauth2_legacy_cbc_decrypt_compat=false in the
proxy bootstrap's global_config static layer, closing the padding oracle
in CVE-2026-47775 for OIDC users by default.

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>
Signed-off-by: Huabing (Robin) Zhao <huabing@tetrate.io>

* docs: note that Replace bootstraps do not get the OAuth2 GCM guards

The guards live in the rendered default bootstrap, which spec.bootstrap
type Replace discards. Call that out in the release note so OIDC users
with a replacement bootstrap know to set them themselves.

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>
Signed-off-by: Huabing (Robin) Zhao <huabing@tetrate.io>

---------

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>
Signed-off-by: Huabing (Robin) Zhao <huabing@tetrate.io>

* fix: list extension manager resources once per reconcile (#9764)

feat: list extension manager resources once per reconcile
Collect ext-GVK in populateExtensionResources and call it once per reconcile from processGateways.

Signed-off-by: stillya <st.ilya.101@gmail.com>

* fix: skip backendResource watch when CRD is missing instead of crashing controller (#9809)

* fix: skip backendResource watch when CRD is missing instead of crashing controller

When an extension manager declares backendResources pointing at a custom
CRD that is absent (or the controller SA lacks list/watch RBAC), the
informer's initial LIST fails, WaitForCacheSync times out, and the
controller crash-loops — taking down the entire gateway, not just the
routes using the custom backend.

This adds a checkCRD call before registering each backendResource watch,
following the same pattern already used for ServiceImport, Backend,
HTTPRouteFilter, and other optional CRDs. If the CRD is missing, the
watch is skipped with a warning log instead of failing cache sync.

Fixes #9807

Signed-off-by: Lucas Siedlarczyk <lucas.siedlarczyk@gmail.com>

* fix merge

Signed-off-by: zirain <zirain2009@gmail.com>

* fix

Signed-off-by: zirain <zirain2009@gmail.com>

---------

Signed-off-by: hai.yue <20416005+yuehaii@users.noreply.github.com>
Signed-off-by: Hai <20416005+yuehaii@users.noreply.github.com>
Signed-off-by: zirain <zirain2009@gmail.com>
Signed-off-by: Jules Dutel <jules.dutel@numberly.com>
Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>
Signed-off-by: Seth Malaki <seth@tigera.io>
Signed-off-by: zanarelli <zanarelli.dev@gmail.com>
Signed-off-by: Andrey Maltsev <maltsev.andrey@gmail.com>
Signed-off-by: Huabing (Robin) Zhao <huabing@tetrate.io>
Signed-off-by: kkk777-7 <kota.kimura0725@gmail.com>
Signed-off-by: stillya <st.ilya.101@gmail.com>
Signed-off-by: Lucas Siedlarczyk <lucas.siedlarczyk@gmail.com>
Co-authored-by: Hai <20416005+yuehaii@users.noreply.github.com>
Co-authored-by: jvlxz <jules.dutel@gmail.com>
Co-authored-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>
Co-authored-by: Seth Malaki <seth.malaki@gmail.com>
Co-authored-by: zanarelli <zanarelli.dev@gmail.com>
Co-authored-by: Andrey Maltsev <maltsev.andrey@gmail.com>
Co-authored-by: Kota Kimura <86363983+kkk777-7@users.noreply.github.com>
Co-authored-by: Starchenko Ilya <st.ilya.101@gmail.com>
Co-authored-by: Lucas Siedlarczyk <lucas.siedlarczyk@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants