[FLINK-40399][state] Separate SavepointKeyFilter runtime contract from push-down planning - #28982
Open
soin08 wants to merge 3 commits into
Open
[FLINK-40399][state] Separate SavepointKeyFilter runtime contract from push-down planning#28982soin08 wants to merge 3 commits into
soin08 wants to merge 3 commits into
Conversation
added 2 commits
August 16, 2026 23:46
…m push-down planning SavepointKeyFilter carried two unrelated responsibilities: the contract the savepoint scan needs, and the algebra used only while combining predicates during filter push-down. The second group was public and @experimental despite being unusable outside push-down translation, to the point that its javadoc had to disclaim it - "Used only while combining filters during push-down translation, not during the scan." Split the two: * org.apache.flink.state.api.filter.SavepointKeyFilter keeps only what the scan needs - test() and getExactKeys() - plus the public factories. * org.apache.flink.state.table.filter.SavepointKeyFilterPlan carries the planning algebra (isEmpty, bounds, intersect, filterKeys, empty), next to the connector that is its only user. SavepointKeyFilterPlan and BoundInfo are public only so the connector in org.apache.flink.state.table can reach them, and are marked @internal. The unit tests are split the same way: the plan algebra cases move to SavepointKeyFilterPlanTest, leaving SavepointFilterTranslatorTest to cover only expression translation. No behaviour change.
Adds cases for savepoint key filter push-down behaviour that had no test, most importantly that predicates which cannot be pushed are returned in remaining() so the runtime still evaluates them. Nothing covered that before: every apply() case asserted remaining() was empty, so a regression there would have silently dropped rows. Also covered: an untranslatable child aborting the whole AND/OR, "key = NULL" whose literal has no readable value, non-Comparable literals on a BYTES key, the two comparison flip directions that were missing, comparison arity, and at SQL level the upper-bound predicates, which had no end-to-end test at all. Finally, every test states the predicate under test as a leading comment, so the SQL shape being exercised is readable without decoding the expression builders.
Collaborator
soin08
commented
Aug 17, 2026
| } | ||
|
|
||
| @Test | ||
| void testOrOfExactAndRangeOnKeyIsNotPushedDownButReturnsCorrectResult() throws Exception { |
Contributor
Author
There was a problem hiding this comment.
this will be supported in the next MR
soin08
commented
Aug 17, 2026
| } | ||
|
|
||
| @Test | ||
| void testOrOfTwoRangesOnKeyIsNotPushedDownButReturnsCorrectResult() throws Exception { |
Contributor
Author
There was a problem hiding this comment.
also will be supported in the next MR
soin08
marked this pull request as ready for review
August 17, 2026 12:46
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What is the purpose of the change
SavepointKeyFiltercarries two unrelated responsibilities: the contract the savepoint scan needs (test,getExactKeys), and the algebra used only while combining predicates during filter push-down (isEmpty,getLowerBound,getUpperBound,intersect,filterKeys,empty, plusBoundInfo).The second group is public despite being unusable outside push-down translation, to the point that its javadoc has to disclaim it: "Used only while combining filters during push-down translation, not during the scan." Anyone implementing the interface for the DataStream API has to read past six methods that do not apply to them.
This separates the two. There is no behaviour change.
Brief change log
org.apache.flink.state.api.filter.SavepointKeyFilterkeeps only the runtime contract,test(K)andgetExactKeys(), plus theexact(..)/range(..)factories.@Internal SavepointKeyFilterPlaninorg.apache.flink.state.table.filter, next to the savepoint connector that is its only user.BoundInfoandEmptyKeyFiltermove with it, andRangeKeyFiltersheds the intersection logic it only needed for push-down.SavepointFilterTranslatoris retargeted to the new type.SavepointKeyFilterPlanTest, leavingSavepointFilterTranslatorTestto cover only expression translation.docs/content/docs/libs/state_processor_api.mdand itscontent.zhcounterpart drop theSavepointKeyFilter.empty()factory and the reference to "remaining interface methods", neither of which exists on the public interface any more.Verifying this change
This change is already covered by existing tests, such as
SavepointFilterTranslatorTest,SavepointDynamicTableSourceTestandSavepointReaderKeyedStateITCase, which exercise filter push-down end to end and are unchanged in behaviour.It also adds tests for behaviour that had no coverage:
remaining()so the runtime still evaluates them. Nothing covered this before: everyapply()case assertedremaining()was empty, so a regression here would have silently returned extra rows.AND/ORrather than being skipped.key = NULL, whose literal has no readable value.Comparableliterals on aBYTESkey, covering the two guards infromBetweenandfromComparisonthat no test reached.GTE,LT), and comparison arity.<and<=, which had no end-to-end test at all, plusANDof two ranges and a literal-on-the-left comparison.Test counts after this PR:
SavepointFilterTranslatorTest54,SavepointKeyFilterPlanTest9 (new),SavepointDynamicTableSourceTest19. Every test now states the predicate under test as a leading comment, so the SQL shape being exercised is readable without decoding the expression builders.Does this pull request potentially affect one of the following parts:
@Public(Evolving): no.SavepointKeyFilteris@Experimental. Six methods are removed from it and moved to an@Internaltype in the connector package; implementers only ever neededtestandgetExactKeys, both unchanged.Documentation
contentandcontent.zhto drop the removed factory method and the stale note about internal interface methods.Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Claude Opus 5)