[IOTDB-17610] Fix LIKE predicate with dynamic pattern expressions#17691
Open
DaZuiZui wants to merge 2 commits into
Open
[IOTDB-17610] Fix LIKE predicate with dynamic pattern expressions#17691DaZuiZui wants to merge 2 commits into
DaZuiZui wants to merge 2 commits into
Conversation
…ttern # Conflicts: # integration-test/src/test/java/org/apache/iotdb/pipe/it/dual/treemodel/auto/basic/IoTDBPipeReceiverAutoCreateDisabledIT.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes #17610.
Content1: Support dynamic pattern expressions in LIKE predicates
This PR allows table-model
LIKEpredicates to accept pattern expressions that are evaluated at runtime, such as:Before this change, the analyzer/planner path did not correctly handle non-literal pattern expressions on the right side of
LIKE. This PR keeps dynamicLIKEpatterns in the normal expression planning/evaluation path, so they are evaluated consistently with other scalar expressions.Design choice: dynamic patterns are handled in the relational analyzer/planner flow instead of adding special execution-only logic.
Alternative considered: fold or rewrite all
LIKEpatterns during analysis. That is only safe for constant patterns, so dynamic expressions should stay in the regular expression path.Content2: Keep metadata predicate pushdown conservative
Metadata predicate conversion remains limited to predicates that can be safely represented as schema filters. Dynamic
LIKEpatterns are not pushed into metadata filters because their final values are unknown during metadata predicate conversion.Design choice: only literal-safe
LIKEpredicates are converted for metadata pushdown; dynamicLIKEexpressions remain normal filters.Alternative considered: evaluate simple functions such as
concat(...)during metadata conversion. That would introduce partial function evaluation into schema predicate conversion and could become inconsistent with the normal evaluator.Content3: Stabilize related load-tsfile scheduling path
This PR also adjusts the related load-tsfile planning/scheduling path needed by the affected dual-tree auto basic CI scenario. The change keeps the existing load node and scheduler organization, preserving the required load-node information across planning and scheduling.
Design choice: extend the existing load-tsfile node/scheduler flow instead of introducing a new wrapper node.
Alternative considered: add a new scheduling wrapper node. That would add extra visitor and node handling for a narrow case, while the existing load node flow is sufficient.
This PR has:
Key changed/added classes (or packages if there are too many classes) in this PR
PredicatePushIntoMetadataChecker: keeps metadata pushdown checks safe for non-literalLIKEpatterns.ConvertSchemaPredicateToFilterVisitor: avoids converting unsupported dynamicLIKEpatterns into schema filters.RelationPlanner: preserves dynamicLIKEexpressions in the relational planning path.AnalyzerTest: adds coverage forLIKE concat(...),LIKE ... || ..., and Chinese string patterns.LoadTsFileNode,LoadSingleTsFileNode,LoadTsFileScheduler,LogicalPlanVisitor: adjust load-tsfile planning/scheduling behavior for the related CI path.LoadTsFileNodeTest: updates load-node test coverage.IoTDBPipeReceiverAutoCreateDisabledIT: covers the related dual-tree auto basic integration path.Tests
./mvnw spotless:apply -pl iotdb-core/datanode,integration-test -P with-integration-tests -ntp git diff --check ./mvnw test -pl iotdb-core/datanode -am -Dtest=AnalyzerTest#expressionTest -DfailIfNoTests=false -Dsurefire.failIfNoSpecifiedTests=false -ntp ./mvnw verify -DskipUTs -Dit.test=IoTDBPipeReceiverAutoCreateDisabledIT -DfailIfNoTests=false -Dfailsafe.failIfNoSpecifiedTests=false -pl integration-test -am -P with-integration-tests -P MultiClusterIT2DualTreeAutoBasic -ntp ./mvnw package -P with-integration-tests -DskipTests -ntpManual verification on a packaged standalone IoTDB cluster:
All three queries returned the expected rows.