Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Surface internal optimization modes (all rows vs first rows) at the SQL and configuration levels #7405

Merged
merged 38 commits into from Sep 13, 2023

Conversation

dyemanov
Copy link
Member

@dyemanov dyemanov commented Dec 1, 2022

The idea is to force the optimizer to consider another (hopefully better) plan if only a subset or rows if fetched initially. For example, this is common for Delphi applications to load only the first records to the grid and continue fetching the following ones on demand (or in background). This feature may improve the response time in such applications. This was internally supported since v3 (used under the hood for FIRST queries, and also for EXISTS subqueries), now these hints may be used for user queries as well.

Syntax OPTIMIZE FOR {ALL | FIRST} ROWS is taken from DB2 which also supports this feature. In fact DB2 allows to specify how many first rows should be taken into account by the optimizer, but I think it complicates its usage, so this is not supported by this patch. Although we could also add it if it's considered useful.

I'm not really happy with new conflicts in the parser, but making OPTIMIZE a reserved keyword is IMO not a good thing either.

@dyemanov dyemanov self-assigned this Dec 1, 2022
@livius2
Copy link

livius2 commented Dec 7, 2022

Hi.
Shouldn't it be settings by RDB$GET_CONTEXT? We can then turn it On before some statements and Off before another.
Or maybe its time to introduce sql hints? Hints will be more usefull as RDB$GET_CONTEXT need roundtrip.

PS. but maybe i simply misunderstood this patch features.

@dyemanov
Copy link
Member Author

dyemanov commented Dec 7, 2022

It's kinda hint, but a very high-level one. It does not suggest the optimizer what exactly to do and you don't need to understand the plan under the hood. It only asks to use a different cost algorithm. As for RDB$CONTEXT, we never used them to turn something on or off. However, a session-level control statement may be useful for this feature to define per-attachment behaviour.

@@ -1344,6 +1344,7 @@ class SelectNode final : public TypedNode<StmtNode, StmtNode::TYPE_SELECT>
bool dsqlForUpdate = false;
bool dsqlWithLock = false;
bool dsqlSkipLocked = false;
TriState dsqlOptimize;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would look for informative some name like dsqlOptimizeForFirst as a bool here has no meaning.

dyemanov and others added 23 commits September 4, 2023 19:16
* WIP

* Original (circa 2022) implementation of the IN LIST optimization, with some post-fixes and minor adjustments

* Make it possible to optimize IN <list> for middle segments in compund indices

* Avoid modifying the retrieval structure at runtime, it may be shared among concurrent requests

* Simplify the code a little. Better cost calculation. Support both root-based and sibling-based list scans inside the same plan node.

* Removed the unneeded const casts and other changed as suggested by Adriano
While this commit does not fix all problems with cached triggers, I see no good reason to defer it.
…. Usually this should not happen, but possible after error thrown during commit (while journal/replica flushing).
…ans. Fixed crash when all IN values are NULLs. Exclude duplicated IN values from the index scan.
… a blob page is not set

1. While performing the intermediate garbage collection.
2. While modifying a blob field of a record.
It can be thrown by lastUsedPage(), when it trying to fetch incorrect page number of PIP
@dyemanov dyemanov merged commit f239ca1 into master Sep 13, 2023
41 of 42 checks passed
@dyemanov dyemanov deleted the optimize-for branch September 13, 2023 18:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

8 participants