-
Notifications
You must be signed in to change notification settings - Fork 3.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
release-24.1: opt: add GenerateTrigramSimilarityInvertedIndexScans rule #122838
release-24.1: opt: add GenerateTrigramSimilarityInvertedIndexScans rule #122838
Conversation
This commit adds a new session setting called `optimizer_use_trigram_similarity_optimization`. It currently has no effect, but in future commits it will allow control over a new optimization for queries with trigram similarity filters. Release note: None
The `GenerateInvertedIndexScans` rule is now disabled for similarity filters on trigram indexes when the `optimizer_use_text_similarity_optimization` session setting is enabled. In a future commit this setting will enable a new rule that makes `GenerateInvertedIndexScans` obsolete. There is no reason to trigger both rules. Release note: None
The `GenerateTrigramSimilarityInvertedIndexScans` exploration rule has been added, which index-accelerates trigram similarity filters. See the comment above the newly added custom function for more details. In future commits, this rule will be optimized further. The tests for this rule break from convention: they are added to a new file rather than to the `select` test file corresponding to the location of the rule in `select.opt`. This proposed new method of organization is motivated by the huge growth of the `select` file over the years. If there is agreement on this, we can incrementally reorganize all existing tests to match. Release note (performance improvement): More efficient query plans are now generated for queries with text similarity filters, e.g., `text_col % 'foobar'`. These plans are generated if the `optimizer_use_trigram_similarity_optimization` session setting is enabled. It is disabled by default.
The `GenerateSimilarityInvertedIndexScans` exploration rule now generates plans that scan fewer trigrams. See the added code comments for more details. Release note: None
Thanks for opening a backport. Please check the backport criteria before merging:
If your backport adds new functionality, please ensure that the following additional criteria are satisfied:
Also, please add a brief release justification to the body of your PR to justify this |
Your pull request contains more than 1000 changes. It is strongly encouraged to split big PRs into smaller chunks. 🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 8 of 8 files at r1, 2 of 2 files at r2, 14 of 14 files at r3, 9 of 9 files at r4, all commit messages.
Reviewable status:complete! 1 of 0 LGTMs obtained (waiting on @michae2)
Backport 4/5 commits from #121973.
/cc @cockroachdb/release
sql: add optimizer_use_trigram_similarity_optimization setting
This commit adds a new session setting called
optimizer_use_trigram_similarity_optimization
. It currently has noeffect, but in future commits it will allow control over a new
optimization for queries with trigram similarity filters.
Release note: None
sql: disable inverted scans when similarity optimization is enabled
The
GenerateInvertedIndexScans
rule is now disabled for similarityfilters on trigram indexes when the
optimizer_use_text_similarity_optimization
session setting is enabled.In a future commit this setting will enable a new rule that makes
GenerateInvertedIndexScans
obsolete. There is no reason to triggerboth rules.
Release note: None
opt: add GenerateTrigramSimilarityInvertedIndexScans rule
The
GenerateTrigramSimilarityInvertedIndexScans
exploration rule hasbeen added, which index-accelerates trigram similarity filters. See the
comment above the newly added custom function for more details. In
future commits, this rule will be optimized further.
The tests for this rule break from convention: they are added to a new
file rather than to the
select
test file corresponding to the locationof the rule in
select.opt
. This proposed new method of organization ismotivated by the huge growth of the
select
file over the years. Ifthere is agreement on this, we can incrementally reorganize all existing
tests to match.
Release note (performance improvement): More efficient query plans are
now generated for queries with text similarity filters, e.g.,
text_col % 'foobar'
. These plans are generated if theoptimizer_use_trigram_similarity_optimization
session setting isenabled. It is disabled by default.
opt: reduce scanned trigrams for similarity filters
The
GenerateSimilarityInvertedIndexScans
exploration rule nowgenerates plans that scan fewer trigrams. See the added code comments
for more details.
Release note: None
Fixes #112675
Release note: None
Release justification: Performance improvements gated behind a
session setting.