adapter: move a replacement's expression cache entries to its target on apply - #38682
Draft
ggevay wants to merge 2 commits into
Draft
adapter: move a replacement's expression cache entries to its target on apply#38682ggevay wants to merge 2 commits into
ggevay wants to merge 2 commits into
Conversation
Applying a materialized view replacement changes an item's definition while the item keeps its GlobalIds. The invalidation on apply only reaches cache entries under the applying process's build version, so a 0dt deployment that cached the pre-apply expressions under its own build version during its read-only phase reads them again after promotion: bootstrap panics with "catalog out of sync" if an old dependency was dropped, and otherwise the old query can be rendered and served. Record the latest RelationVersion of the owning item with each entry and drop entries recorded at another version when the cache is opened. The apply bumps the target's version, so stale entries are dropped whoever wrote them. Tests: a unit test for the version check, a server test that reproduces the bootstrap panic without the fix, and a platform check that applies a replacement and drops its old input in the first manipulate phase, which the zero-downtime upgrade scenarios run inside the rollout window. Closes: SQL-683 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…on apply After a successful apply, the cache moves the replacement's own entries to the surviving item's ids at its new version and removes the item's other entries, so the next boot hits instead of re-optimizing the view. The moved entries keep the optimizer features that produced them, so a feature change between the replacement's creation and the apply makes the next boot re-optimize the view like any other item. Nothing depends on the write landing: entries recorded before the apply carry the old version and the next open drops them either way. The cache open now logs how many entries it kept and removed, by reason, and drops entries the running build cannot read instead of reporting them on every boot. Tests: a unit test for the move, and a server test that creates an index on the replacement's input after the replacement was planned and checks that the boot after the apply still reads the input directly, which a re-optimization would not. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
ggevay
force-pushed
the
gabor/sql-683-expr-cache-restamp-on-apply
branch
from
September 5, 2026 18:48
66ea566 to
8f64fc9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is on top of #38677, so please review the "adapter: record the item version in expression cache entries" commit there.
Motivation
#38677 fixes SQL-683 by having
ExpressionCache::opendrop entries recorded for an earlier definition of an item. After anALTER MATERIALIZED VIEW ... APPLY REPLACEMENTthat leaves the next boot re-optimizing the replaced materialized view once, and nothing in the logs says why an entry was dropped. This PR is stacked on #38677 (its first commit); review the second commit.Description
After a successful apply, the coordinator asks the cache to move the replacement's own entries to the surviving item: the local entry moves under the item's root id, the global entry stays under the writing id, both are recorded at the new version, and the item's other entries are removed. The moved entries keep the optimizer features that produced them, so a feature change between the replacement's creation and the apply makes the next boot re-optimize the view like any other item. Correctness does not rest on this write, the version check in #38677 does that; it only spares the boot after an apply one re-optimization per applied replacement.
The move runs inside the cache task (
ExpressionCache::restamp), reading the replacement's entries from the durable cache's local mirror. An entry that is unreadable, or missing from this process's view of the cache, is skipped and removed, leaving the next open to re-optimize what it would have served. The read and the rewrite are not atomic against another process of the same build version, which is safe because ids are never reused: whatever such a process writes under the replacement's id describes the definition being adopted.ExpressionCache::opennow logs how many entries it kept and how many it removed, by reason (dropped item, item version mismatch, dropped index import, unreadable, prior build version), so a cache miss can be attributed after the fact. An entry the running build cannot read is dropped instead of being reported on every boot; entries are only ever read by the build version that wrote them, so this can only affect entries the same build wrote.Tests
expr_cacheunit test: the moved entries of an apply are returned under the target's ids at the new version, and the entries the move invalidates are gone even where the version check would have kept them.test_replacement_materialized_view_expression_cache_hit_after_apply(environmentd server tests): creates an index on the replacement's input after the replacement was planned, applies, restarts, and checks throughmz_compute_dependenciesthat the view's dataflow still reads the input directly. A re-optimization at boot adopts the index, which is what happens without this change.Alternatives
Writing the surviving item's in-memory plans to the cache after the apply, stamped with the cluster's current optimizer features, would avoid the re-optimization without a new cache operation. But the apply moves the replacement's plans over without re-optimizing, so a feature change between the replacement's creation and the apply would stamp an old-feature plan with the new features, and every later boot would hit and reinstall it while re-optimizing everything else on the cluster. Moving the replacement's own entries keeps the features that produced each plan.
Part of SQL-683.
🤖 Generated with Claude Code