sql: better logging about the optimizer as well as some cancellation checks #85041
Conversation
|
This change is |
yuzefovich
force-pushed
the
opt-tracing
branch
2 times, most recently
from
5bc8e7b
to
6e3f944
Compare
|
Rebased on top of master, RFAL. |
yuzefovich
force-pushed
the
opt-tracing
branch
2 times, most recently
from
e02ed2c
to
7b5b2ed
Compare
There was a problem hiding this comment.
This is great, thanks for doing this!
Reviewed 16 of 18 files at r1, 7 of 7 files at r3, 5 of 5 files at r4, all commit messages.
Reviewable status:complete! 1 of 0 LGTMs obtained (waiting on @rytaft and @yuzefovich)
-- commits line 20 at r4:
nit: make this more specific, like:
This situation is now improved by periodically checking for the context cancellation every time a group is optimized by optimizeGroup. It's still possible for the optimizer to continue running long after cancellation if there is a long running procedure at a deeper level. For example, a long-running, individual invocation of an exploration rule will not halt due to context cancellation. However, the cancellation improvement in this commit should halt long-running optimization in most cases.
This commit derives a separate tracing span on the main query path that covers the whole optimizer as well as adds a few logging messages around the optbuilder and the optimization. It also audits some of the already present logs to be written down before performing possibly long operations. Release note: None
|
Fixed the commit message and moved some changes from the second commit into the first one so that it compiles. TFTR! bors r+ |
Previously, the optimizer didn't check the context cancellation ever which could lead to surprising behavior - e.g. if the statement timeout occurs in the middle of the optimization, we would still finish the whole optimization phase only to realize that we were canceled at the execution time. This situation is now improved by periodically checking for the context cancellation every time a group is optimized by `optimizeGroup`. It's still possible for the optimizer to continue running long after cancellation if there is a long running procedure at a deeper level. For example, a long-running, individual invocation of an exploration rule will not halt due to context cancellation. However, the cancellation improvement in this commit should halt long-running optimization in most cases. Release note: None
|
Canceled. |
|
Forgot to include the new logic test file, now fixed. bors r+ |
|
Build succeeded: |
sql: add logging about the optimizer into the trace
This commit derives a separate tracing span on the main query path that
covers the whole optimizer as well as adds a few logging messages around
the optbuilder and the optimization. It also audits some of the already
present logs to be written down before performing possibly long
operations.
Release note: None
xform: check for context cancellation
reviously, the optimizer didn't check the context cancellation ever
which could lead to surprising behavior - e.g. if the statement timeout
occurs in the middle of the optimization, we would still finish the
whole optimization phase only to realize that we were canceled at the
execution time.
This situation is now improved by periodically checking for the context
cancellation every time a group is optimized by
optimizeGroup. It'sstill possible for the optimizer to continue running long after
cancellation if there is a long running procedure at a deeper level.
For example, a long-running, individual invocation of an exploration
rule will not halt due to context cancellation. However,
the cancellation improvement in this commit should halt long-running
optimization in most cases.
Fixes: #70245.
Addresses: #70314.
Release note: None