Fix duplicate results and failures of queries with indexes and query parameters and improve index scan generation [HZ-3014] [HZ-3013] #25527
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
k-jamroz
changed the title
k-jamroz
added
Type: Defect
Source: Internal
|
The job Click to expand the log file---------ERRORS----------- -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-builder_2/hazelcast/src/main/java/com/hazelcast/internal/iteration/IndexIterationPointer.java:149:84: '&&' should be on a new line. [OperatorWrap] -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-builder_2/hazelcast/src/main/java/com/hazelcast/internal/iteration/IndexIterationPointer.java:272:13: Name 'rf_lt' must match pattern '^[a-z][a-zA-Z0-9]*$'. [LocalVariableName] -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-builder_2/hazelcast/src/test/java/com/hazelcast/internal/iteration/IndexIterationPointerTest.java:1: File does not end with a newline. [NewlineAtEndOfFile] -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-builder_2/hazelcast/src/test/java/com/hazelcast/internal/iteration/IndexIterationPointerTest.java:1: Line does not match expected header line of '/*'. [Header] -------------------------- [ERROR] Failed to execute goal org.apache.maven.plugins:maven-checkstyle-plugin:3.3.0:checkstyle (default) on project hazelcast: An error has occurred in Checkstyle report generation. Failed during checkstyle execution: There are 4 errors reported by Checkstyle 9.3 with /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-builder_2/checkstyle/checkstyle.xml ruleset. -> [Help 1] -------------------------- [ERROR] -------------------------- [ERROR] Re-run Maven using the -X switch to enable full debug logging. -------------------------- [ERROR] -------------------------- [ERROR] For more information about the errors and possible solutions, please read the following articles: -------------------------- [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException -------------------------- [ERROR] -------------------------- [ERROR] After correcting the problems, you can resume the build with the command -------------------------- [ERROR] mvn -rf :hazelcast -------------------------- |
|
The job Click to expand the log file---------ERRORS----------- -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-compiler/hazelcast/src/main/java/com/hazelcast/internal/iteration/IndexIterationPointer.java:149:84: '&&' should be on a new line. [OperatorWrap] -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-compiler/hazelcast/src/main/java/com/hazelcast/internal/iteration/IndexIterationPointer.java:272:13: Name 'rf_lt' must match pattern '^[a-z][a-zA-Z0-9]*$'. [LocalVariableName] -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-compiler/hazelcast/src/test/java/com/hazelcast/internal/iteration/IndexIterationPointerTest.java:1: File does not end with a newline. [NewlineAtEndOfFile] -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-compiler/hazelcast/src/test/java/com/hazelcast/internal/iteration/IndexIterationPointerTest.java:1: Line does not match expected header line of '/*'. [Header] -------------------------- [ERROR] Failed to execute goal org.apache.maven.plugins:maven-checkstyle-plugin:3.3.0:checkstyle (default) on project hazelcast: An error has occurred in Checkstyle report generation. Failed during checkstyle execution: There are 4 errors reported by Checkstyle 9.3 with /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-compiler/checkstyle/checkstyle.xml ruleset. -> [Help 1] -------------------------- [ERROR] -------------------------- [ERROR] Re-run Maven using the -X switch to enable full debug logging. -------------------------- [ERROR] -------------------------- [ERROR] For more information about the errors and possible solutions, please read the following articles: -------------------------- [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException -------------------------- [ERROR] -------------------------- [ERROR] After correcting the problems, you can resume the build with the command -------------------------- [ERROR] mvn -rf :hazelcast -------------------------- |
k-jamroz
force-pushed
the
normalize-index-pointers
branch
from
c27334a
to
3b65d7a
Compare
|
The job Click to expand the log file---------ERRORS----------- -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-builder/hazelcast-sql/src/main/java/com/hazelcast/jet/sql/impl/connector/map/QueryUtil.java:174:110: ')' is preceded with whitespace. [ParenPad] -------------------------- [ERROR] Failed to execute goal org.apache.maven.plugins:maven-checkstyle-plugin:3.3.0:checkstyle (default) on project hazelcast-sql: An error has occurred in Checkstyle report generation. Failed during checkstyle execution: There is 1 error reported by Checkstyle 9.3 with /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-builder/checkstyle/checkstyle.xml ruleset. -> [Help 1] -------------------------- [ERROR] -------------------------- [ERROR] Re-run Maven using the -X switch to enable full debug logging. -------------------------- [ERROR] -------------------------- [ERROR] For more information about the errors and possible solutions, please read the following articles: -------------------------- [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException -------------------------- [ERROR] -------------------------- [ERROR] After correcting the problems, you can resume the build with the command -------------------------- [ERROR] mvn -rf :hazelcast-sql -------------------------- |
k-jamroz
force-pushed
the
normalize-index-pointers
branch
from
09aef32
to
ca9f950
Compare
k-jamroz
changed the title
k-jamroz
force-pushed
the
normalize-index-pointers
branch
from
ef2ace9
to
ea4a440
Compare
k-jamroz
force-pushed
the
normalize-index-pointers
branch
from
ea4a440
to
3640c6c
Compare
Fly-Style
approved these changes
Oct 5, 2023
| // both ends are equal, but might differ in inclusiveness, | ||
| // use one which covers bigger range | ||
| return tuple2(left, leftInclusive || rightInclusive); | ||
| } else if (result < 0) { |
There was a problem hiding this comment.
nit: here you can use ternary operator, if you like it.
| // both ends are equal, but might differ in inclusiveness, | ||
| // use one which covers bigger range | ||
| return tuple2(left, leftInclusive || rightInclusive); | ||
| } else if (result < 0) { |
ivanthescientist
approved these changes
Oct 10, 2023
| @@ -142,9 +149,14 @@ private static IndexComponentFilter convertFromRangeFilters( | |||
| toInclusive = candidateFilter.isToInclusive(); | |||
| expressions.add(candidate.getExpression()); | |||
| } | |||
|
|
|||
| if (from == null && to == null && candidateFilter.getFrom() == null && candidateFilter.getTo() == null) { | |||
There was a problem hiding this comment.
Not related to this PR, but feels like all these specific condition checks to handle NULL should live somewhere outside of this class or alternatively be moved to their own separate methods to convey logic more clearly, although maybe comments would work too.
There was a problem hiding this comment.
the null/NULL checks generally feel very inconsistent, especially in different layers (optimizer, DAG, Index and IndexStore, Predicate API has also its own logic). I had lot's of headaches because of that but I did not feel confident enough to make it more consistent. Sometimes null means NULL, sometimes unbounded (including NULL or not), sometimes it is invalid... (probably somewhere is also means NOT NULL - for range filters)
|
run-ee-tests |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In order to generate correct results for queries with query parameters normalisation of index iteration pointers set is performed to make it disjoint and ordered. This has to be done late, when query parameter values are known (during execution not optimisation), for literals this happens during query optimisation. This process uses special treatment of
NULLvalue which can be compared and satisfies the following ordering:-Inf < NULL < non null value < +Inf. This ordering is compatible with SQLORDER BY x NULLS FIRSTandORDER BY x DESC NULLS LAST- this is the only currently available NULL placement in ordered results.After that some obsolete restrictions for index usage (e.g. disjunction of ranges) could be removed and
IS NOT NULLpredicates can also be correctly executed using index.Fixes HZ-3013, HZ-3014
Checklist:
Team:,Type:,Source:,Module:) and Milestone setAdd to Release NotesorNot Release Notes contentset