Skip to content

router: keep ECDS filter config provider manager alive for its subscriptions - #46538

Merged
yanavlasov merged 1 commit into
envoyproxy:mainfrom
adelsam:fix/ecds-provider-manager-lifetime
Aug 4, 2026
Merged

router: keep ECDS filter config provider manager alive for its subscriptions#46538
yanavlasov merged 1 commit into
envoyproxy:mainfrom
adelsam:fix/ecds-provider-manager-lifetime

Conversation

@adelsam

@adelsam adelsam commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Commit Message: router: keep ECDS filter config provider manager alive for its subscriptions

Additional Description:

Filter::FilterConfigSubscription stores its provider manager as a raw reference and writes through
it from its destructor:

// source/common/filter/config_discovery_impl.h
FilterConfigProviderManagerImplBase& filter_config_provider_manager_;

// source/common/filter/config_discovery_impl.cc
FilterConfigSubscription::~FilterConfigSubscription() {
  init_target_.ready();
  filter_config_provider_manager_.subscriptions_.erase(subscription_id_);
}

The manager must therefore outlive every subscription it created. The upstream HTTP filter config
provider manager is a singleton registered without pin, so Singleton::ManagerImpl retains only a
weak_ptr and the documented contract in envoy/singleton/manager.h applies:

All code that uses singletons must store the shared_ptr for as long as the singleton is needed.

ClusterInfoImpl (upstream_impl.h) and UdpProxyFilterConfigImpl (udp_proxy/config.h) comply,
each holding the manager as a member declared before their filter factory lists.
Router::FilterConfig did not: it obtained the manager as a constructor-local shared_ptr and
dropped it on return, while the ECDS subscriptions created by processFilters() live on in
upstream_http_filter_factories_. When no other holder exists at that moment, the manager is
destroyed as the constructor returns and those subscriptions hold a dangling reference for the
remaining life of the process, dereferencing it on teardown.

ClusterInfoImpl creates this manager unconditionally, so an existing cluster normally masks the
defect. The reachable window is a router config built while no cluster exists — for example a
statically configured listener with CDS-supplied clusters.

This change retains the manager in a member declared before upstream_http_filter_factories_, so
reverse-declaration destruction tears down the providers and their subscriptions first and releases
the manager last, matching the existing ClusterInfoImpl and UDP proxy pattern.

Composite's ExecuteFilterAction has the same constructor-local pattern for dynamic_config actions
and is hardened identically. That instance is currently masked by ClusterInfoImpl holding the same
singleton, so it is defense-in-depth rather than a live defect.

Risk Level: Low — adds one member and promotes one constructor-local to it. No logic, ordering, or
behavior change on any request path.

Testing:

New regression test RouterUpstreamFilterTest.DynamicFilterKeepsConfigProviderManagerAlive.

Configuration Result
Pre-fix, no sanitizer fails (manager.expired() is true)
Post-fix, no sanitizer passes
Pre-fix, ASAN heap-use-after-free (below)
Post-fix, ASAN passes, no sanitizer findings

//test/extensions/filters/http/composite:filter_test passes (60/60), covering the composite change.

AddressSanitizer against the unmodified code:

==2818==ERROR: AddressSanitizer: heap-use-after-free on address 0x60b000012558
READ of size 8 at 0x60b000012558 thread T0
    #0 absl::container_internal::raw_hash_set<FlatHashMapPolicy<std::string,
         std::weak_ptr<Envoy::Filter::FilterConfigSubscription>>>::AssertNotDebugCapacity()
    #1 ...::find<std::string>(std::string const&)
    #2 Envoy::Filter::FilterConfigSubscription::~FilterConfigSubscription()
    #4 Envoy::Filter::DynamicFilterConfigProviderImplBase::~DynamicFilterConfigProviderImplBase()
    #7 std::vector<Envoy::Http::FilterChainUtility::FilterFactoryProvider>::__destroy_vector
    #8 Envoy::Router::FilterConfig::~FilterConfig()

previously allocated by thread T0 here:
    #4 Envoy::Http::FilterChainUtility::createSingletonUpstreamFilterConfigProviderManager(...)
    #5 Envoy::Router::FilterConfig::FilterConfig(...)

The container in frame #0 is FilterConfigProviderManagerImplBase::subscriptions_.

One detail stated precisely: the "freed by" frame in this trace is the test's own re-lookup of the
singleton, not the constructor. Because the manager is created via make_shared, the object is
destructed when the constructor's local reference drops — running ~flat_hash_map and releasing the
map's internal buffer — while the enclosing block survives until Singleton::ManagerImpl overwrites
its stale weak_ptr. Production therefore hits use-after-destruction of the same member slightly
earlier than this trace's free point. Same dangling reference, same container, different moment.

Sanitizer runs were performed on macOS/arm64 with Apple clang, because --config=asan currently
hardcodes an x86-64 Linux runtime path. Equivalent flags were used: --copt=-fsanitize=address --copt=-fno-omit-frame-pointer --copt=-O1 --linkopt=-fsanitize=address --strip=never --dynamic_mode=off.

Docs Changes: n/a

Release Notes: added changelogs/current/bug_fixes/router__upstream-http-filter-config-provider-manager-lifetime.rst

Platform Specific Features: n/a


Per the generative AI policy:
this investigation, patch, and test were produced with AI assistance (Claude). I have reviewed and
understand the change.

…ions

Filter::FilterConfigSubscription stores the provider manager as a raw
reference and writes through it from its destructor:

  FilterConfigSubscription::~FilterConfigSubscription() {
    init_target_.ready();
    filter_config_provider_manager_.subscriptions_.erase(subscription_id_);
  }

The manager must therefore outlive every subscription it created. Most
owners get this right (ClusterInfoImpl and UdpProxyFilterConfigImpl both
declare the manager before the factory lists; the listener-side managers
live in ProdListenerComponentFactory, which outlives the listener lists),
but two call sites obtain the manager as a local shared_ptr and drop it:

- Router::FilterConfig, for `upstream_http_filters`. The upstream manager
  singleton is registered *unpinned*, so Singleton::ManagerImpl only holds
  a weak_ptr. Its sole other holder is ClusterInfoImpl. When the router
  config is built while no cluster exists -- a static listener with
  CDS-only clusters, for example -- the local reference is the only one,
  and the manager dies at the end of the constructor. Every subscription
  created for those filters then holds a dangling reference for the life
  of the process and writes through it at teardown, corrupting the heap.

- Composite's ExecuteFilterAction, for `dynamic_config` actions. Same
  pattern; currently masked for the upstream variant by ClusterInfoImpl
  happening to hold the same singleton.

Fix both by retaining the manager, declared so that it is destroyed after
the providers whose subscriptions reference it.

Note that ClusterInfoImpl destruction is deferred via
Dispatcher::deleteInDispatcherThread, and that queue is not drained until
dispatcher_->shutdown() -- after listener_manager_.reset() in
~InstanceBase. Plain server shutdown is therefore safe today only by
accident of that ordering, not by design.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Samuel Adelman <samuel.adelman@gmail.com>
@jmarantz

jmarantz commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

See also #46439

@yanavlasov
yanavlasov enabled auto-merge (squash) August 4, 2026 14:47
@yanavlasov
yanavlasov merged commit f17808c into envoyproxy:main Aug 4, 2026
26 of 27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants