Skip to content

Add support for envoy gateway#5150

Merged
smatting merged 79 commits into
developfrom
WPB-23903-routes-1
May 11, 2026
Merged

Add support for envoy gateway#5150
smatting merged 79 commits into
developfrom
WPB-23903-routes-1

Conversation

@smatting
Copy link
Copy Markdown
Contributor

@smatting smatting commented Mar 23, 2026

This PR:

  • Introduces a new Helm chart wire-ingress that targets Envoy Gateway. It is intended as a replacement for the nginx-ingress-services chart, which uses ingress-nginx. The wire-ingress chart is not production-ready yet.

  • Changes the integration test suite: all tests now run against the wire-ingress chart. The ingress solution can be selected via the WIRE_INGRESS_MODE environment variable. The federation domains change in Envoy mode — see comments in the code. Changes to the federator and integration charts are made to accommodate both variants for
    testing. As a consequence, any changes to nginx-ingress-services will be untested once this PR is merged. I've added a check integration-setup-federation.sh that prevents any changes to nginx-ingress-services to avoid this being overlooked.

  • Changes the temporary filenames used in the integration test suite. This fixes issues with filenames that were too long for nginz to handle.

  • Deletes the unused file hack/helmfile-federation-v0.yaml.gotmpl.

  • Add a post-upgrade to all objects needed for testing. This makes running tests on the cluster manually more convenient

Checklist

  • Revert .envrc before merging !!!!
  • Add a new entry in an appropriate subdirectory of changelog.d
  • Read and follow the PR guidelines

@smatting smatting force-pushed the WPB-23903-routes-1 branch from c4a1b87 to 28aad07 Compare March 23, 2026 13:59
@zebot zebot added the ok-to-test Approved for running tests in CI, overrides not-ok-to-test if both labels exist label Mar 23, 2026
@wartraxx51
Copy link
Copy Markdown

wartraxx51 commented Mar 24, 2026

Naming thing that bugs me across the whole PR, the chart is wire-ingress, the file is ingress-envoy.yaml the route is nginz-websockets. None of thes have any Ingress in them. My point of view, name it like the replacement wire-gateway , gateway-envoy.yaml etc... not blocking when I open a file called ingress-envoy.yaml and the first thing I see is kind: gateway. I have to stop and double check.

@wartraxx51
Copy link
Copy Markdown

A BackendTrafficPolicy is missing for WebSockets, the default timeouts will terminate long-lived connections, I didn't check the default value, https://gateway.envoyproxy.io/latest/api/extension_types/#backendtrafficpolicy

@smatting smatting force-pushed the WPB-23903-routes-1 branch from 8e81e3f to 83b86e7 Compare March 25, 2026 14:02
@smatting
Copy link
Copy Markdown
Contributor Author

Naming thing that bugs me across the whole PR, the chart is wire-ingress, the file is ingress-envoy.yaml the route is nginz-websockets. None of thes have any Ingress in them. My point of view, name it like the replacement wire-gateway , gateway-envoy.yaml etc... not blocking when I open a file called ingress-envoy.yaml and the first thing I see is kind: gateway. I have to stop and double check.

I changed the ingress-envoy.yaml filename. But I couldn't find other occurences. Everywhere the filenames should be <kind>...
Can you please more concrete?

@smatting
Copy link
Copy Markdown
Contributor Author

smatting commented Mar 25, 2026

A BackendTrafficPolicy is missing for WebSockets, the default timeouts will terminate long-lived connections, I didn't check the default value, https://gateway.envoyproxy.io/latest/api/extension_types/#backendtrafficpolicy

Thank you! Added the policy. We need to run QA tests against this on staging to see if it really works

@smatting smatting marked this pull request as ready for review March 25, 2026 14:26
@smatting smatting requested review from a team as code owners March 25, 2026 14:26
Comment thread charts/wire-ingress/templates/envoyextensionpolicy-federator.yaml
Comment thread charts/wire-ingress/templates/envoyextensionpolicy-federator.yaml
Comment thread charts/integration/templates/envoy-gateway.yaml
name: {{ printf "%s/%s/https" .Release.Namespace $gatewayName | quote }}
operation:
op: add
path: "/filter_chains/0/filters/0/typed_config/strip_trailing_host_dot"
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why don't you use the same approach las in charts/wire-ingress/templates/envoypatchpolicy-federator.yaml:36-41

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, this is a leftover from a previous solution. I switched to the new solution in 05e23ad

Comment thread charts/integration/templates/envoy-gateway.yaml
Comment thread charts/wire-ingress/templates/service-account-pages.yaml
Comment thread charts/wire-ingress/templates/service-team-settings.yaml
Comment thread charts/wire-ingress/templates/service-test-fed.yaml
Comment thread charts/wire-ingress/templates/service-webapp.yaml
Comment thread charts/wire-ingress/templates/service-test-fed.yaml
Comment thread charts/wire-ingress/values.yaml
Comment thread charts/wire-ingress/README.md Outdated
Comment thread charts/wire-ingress/README.md Outdated
controllerName: gateway.envoyproxy.io/gatewayclass-controller
```

You need to refer to this object in the `gateway.className` paramter.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

parameter

| `config.dns.base` | Only used for CSP header rendering, which is a multi-ingress feature |
| `tls.verify_depth` | Envoy Gateway `ClientTrafficPolicy` does not expose a direct verify-depth knob; the CA chain itself controls this |
| `tls.enabled` | Removed — had no effect; all routes are always TLS-terminated |
| `secrets.tlsClientCA` | No longer supplied via values. The `federator-ca` ConfigMap is created by the wire-server chart and referenced directly. |
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is created by federator chart ?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as previous comment

Comment thread charts/wire-ingress/templates/_helpers.tpl
@smatting smatting force-pushed the WPB-23903-routes-1 branch from f8ead45 to a59a6c7 Compare April 29, 2026 09:40
Comment thread charts/wire-ingress/templates/httproute-s3.yaml
Copy link
Copy Markdown
Contributor

@fisx fisx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i've read all the non-helm code (3 files), LGTM!

CHARTS_DIR="${TOP_LEVEL}/.local/charts"
HELM_PARALLELISM=${HELM_PARALLELISM:-1}

changed_files=$(git --no-pager diff-tree --no-commit-id -r --name-only HEAD)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so this guard only works if i don't commit my changes first, before running tests. i guess that's a lot better than no guard. you could compute a sha256sum from all of nginx-ingress-services i guess, and compare against a stored copy of the hash. if there is a mismatch, the error will instruct you to commit the changes and update the hash in the script.

i'm not saying you should do that, this is just my brain having ideas.

Copy link
Copy Markdown
Contributor Author

@smatting smatting May 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there was a misunderstanding, which @fisx resolved in 1on1: the guard actually compares against the merge-base of the PR in the ci pipeline, so it effectively prevents any changes

Comment thread integration/test/Testlib/ModService.hs
@smatting smatting merged commit e1984fc into develop May 11, 2026
10 checks passed
@smatting smatting deleted the WPB-23903-routes-1 branch May 11, 2026 12:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ok-to-test Approved for running tests in CI, overrides not-ok-to-test if both labels exist

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants