Skip to content

Filter Chain DS support #39141

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
4 tasks
ohadvano opened this issue Apr 16, 2025 · 6 comments
Open
4 tasks

Filter Chain DS support #39141

ohadvano opened this issue Apr 16, 2025 · 6 comments
Labels
area/filter_chains area/xds enhancement Feature requests. Not bugs or questions. no stalebot Disables stalebot from closing an issue

Comments

@ohadvano
Copy link
Contributor

ohadvano commented Apr 16, 2025 •
edited
Loading

This is a feature tracking issue for adding filter chain discovery service support in Envoy.

Filter chain discovery can be looked at in two ways:

  1. A filter chain is explicitly specified in the bootstrap configurations, but its configuration is discovered at runtime
  2. Listener's filter chains are not specified in the bootstrap, rather than being discovered at runtime. This can either be all of the listener's filter chains or some of them (a model where some of the filter chains are static and some are runtime-discovered, but they co-exist within the same listener)

Although the goal is achieving both of the capabilities above, first step focuses on the 2nd capability of FCDS.
This feature was mentioned in the past in #4540.

There have been improvements to Envoy's listeners since then. For example, when Envoy receives an LDS update, it only adjusts the filter chains which have been modified, preventing the draining of connections of unaffected filter chains.
While this is useful for most cases, in some other cases this is still not robust enough. In a deployment where a filter chain represents a small scoped system object, a single listener may contain a high number of filter chains. A single change (add/remove/update) requires xDS server to send the entire listener object and Envoy to process that object. If the updates are frequent, this becomes an issue that causes high config update latencies as well as busier main-thread.
Having the ability to stream updates just for a specific filter chain change not only simplifies the deployments and control plane management but also improves how Envoy copes with this kind of use case.

The roadmap for this feature:

  • API changes with unimplemented annotation
  • Add FCDS service implementation, unit tests, integration tests
  • (If needed) Refactoring work in listener_impl
  • Consume FCDS in listener, unit tests, integration tests, documentation

cc @envoyproxy/api-shepherds, @ggreenway
@ohadvano ohadvano added enhancement Feature requests. Not bugs or questions. triage Issue requires triage labels Apr 16, 2025
@ohadvano ohadvano mentioned this issue Apr 17, 2025
Merged
@phlax phlax added area/xds area/filter_chains and removed triage Issue requires triage labels Apr 17, 2025
@yanavlasov
Copy link
Contributor

@adisuissa

@adisuissa
Copy link
Contributor

Here's some FCDS history that I'm aware of:

@ohadvano
Copy link
Contributor Author

ohadvano commented Apr 18, 2025 •
edited
Loading

Thanks @adisuissa, my intention is indeed creating small gradual PRs in the direction of implementing this. The breakdown is described above in the issue description. If you feel there's some missing step please let me know.
The API change is ready for review here: #39154, you're also welcome to review it although you're not assigned to it.
I'm hoping to first close on the API before proceeding to code changes, to make sure I don't waste time on something that won't align with the specs..

@adisuissa
Copy link
Contributor

The breakdown is described above in the issue description. If you feel there's some missing step please let me know.

I think the high-level plan is ok, but will probably need to be broken down further to smaller chunks.

wbpcode pushed a commit that referenced this issue Apr 26, 2025
@ohadvano
fcds: add unimplemented api (#39154)
9ede1da 
Additional Description: Adding unimplemented FCDS API as a first step in
roadmap to support this feature. See #39141 for feature description.
Most of the specifications are described inline with the API changes but
for extra clarity I will describe them here:
- Current FCDS implementation is intended to discover a filter chain
object (including its name)
- It's meant to support both delta and SoTW APIs
- A discovery request should include the containing context name. The
containing context can either be customized per discovery in the config
or derived by the listener's name.
- A listener may contain both statically configured filter chains as
well as runtime discovered filter chains. However, if a discovered
filter chain matches any of another existing filter chain, its
configuration is discarded (this will be later indicated by metrics
and/or logs).
- The model here is very similar to how EDS works. A cluster owns an EDS
subscription. It uses a custom service name or the cluster name in the
discovery request and the server responds with a list of endpoints. In
this discovery service, the listener owns the FCDS subscription,
mentioned by the listener name or custom configured name, and the server
responds with list of filter chains.

Example usage:
```
static_resources:
  listeners:
  - name: tcp_listener_1
    stat_prefix: tcp_listener
    address:
      socket_address:
        protocol: TCP
        address: 0.0.0.0
        port_value: 1111
    filter_chains:
    - name: some_static_filter_chain
      filters:
      - filter_1
      - filter_2
    fcds_config:
      name: tcp_listener_1_dynamic_filter_chains
      config_source:
        api_config_source:
          api_type: GRPC
          transport_api_version: V3
          grpc_services:
          - envoy_grpc:
              cluster_name: xds_cluster
```

Risk Level: low
Testing: none
Docs Changes: API
Release Notes: none
Platform Specific Features: none

---------

Signed-off-by: Ohad Vano <ohadvano@gmail.com>
Signed-off-by: ohadvano <49730675+ohadvano@users.noreply.github.com>
jtlisi pushed a commit to jtlisi/envoy that referenced this issue Apr 28, 2025
@ohadvano @jtlisi
fcds: add unimplemented api (envoyproxy#39154)
c187960 
Additional Description: Adding unimplemented FCDS API as a first step in
roadmap to support this feature. See envoyproxy#39141 for feature description.
Most of the specifications are described inline with the API changes but
for extra clarity I will describe them here:
- Current FCDS implementation is intended to discover a filter chain
object (including its name)
- It's meant to support both delta and SoTW APIs
- A discovery request should include the containing context name. The
containing context can either be customized per discovery in the config
or derived by the listener's name.
- A listener may contain both statically configured filter chains as
well as runtime discovered filter chains. However, if a discovered
filter chain matches any of another existing filter chain, its
configuration is discarded (this will be later indicated by metrics
and/or logs).
- The model here is very similar to how EDS works. A cluster owns an EDS
subscription. It uses a custom service name or the cluster name in the
discovery request and the server responds with a list of endpoints. In
this discovery service, the listener owns the FCDS subscription,
mentioned by the listener name or custom configured name, and the server
responds with list of filter chains.

Example usage:
```
static_resources:
  listeners:
  - name: tcp_listener_1
    stat_prefix: tcp_listener
    address:
      socket_address:
        protocol: TCP
        address: 0.0.0.0
        port_value: 1111
    filter_chains:
    - name: some_static_filter_chain
      filters:
      - filter_1
      - filter_2
    fcds_config:
      name: tcp_listener_1_dynamic_filter_chains
      config_source:
        api_config_source:
          api_type: GRPC
          transport_api_version: V3
          grpc_services:
          - envoy_grpc:
              cluster_name: xds_cluster
```

Risk Level: low
Testing: none
Docs Changes: API
Release Notes: none
Platform Specific Features: none

---------

Signed-off-by: Ohad Vano <ohadvano@gmail.com>
Signed-off-by: ohadvano <49730675+ohadvano@users.noreply.github.com>
Signed-off-by: Jacob Lisi <jtlisi@google.com>
@ohadvano ohadvano mentioned this issue May 12, 2025
Open
@github-actions GitHub Actions
Copy link

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or "no stalebot" or other activity occurs. Thank you for your contributions.

@github-actions github-actions bot added the stale stalebot believes this issue/PR has not been touched recently label May 18, 2025
@ohadvano
Copy link
Contributor Author

not stale

@github-actions github-actions bot removed the stale stalebot believes this issue/PR has not been touched recently label May 18, 2025
@tonya11en tonya11en added the no stalebot Disables stalebot from closing an issue label May 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/filter_chains area/xds enhancement Feature requests. Not bugs or questions. no stalebot Disables stalebot from closing an issue
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@phlax @tonya11en @adisuissa @yanavlasov @ohadvano