sql: prevent illegal concurrency with internal executor in some cases #119176
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![blathers[crl]](https://avatars.githubusercontent.com/in/59700?s=40&u=373d9101a12edc3847d2fb6ae78e3adcfb4ca41e&v=4){kind=avatar}
|
It looks like your PR touches production code but doesn't add or edit any test code. Did you consider adding tests to your PR? 🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf. |
|
This change is |
DrewKimball
approved these changes
Feb 14, 2024
There was a problem hiding this comment.
Reviewed 1 of 1 files at r1, 1 of 1 files at r2, all commit messages.
Reviewable status:complete! 1 of 0 LGTMs obtained (waiting on @rafiss)
Previously, it was possible for some methods of the internal executor
result in illegal concurrency. In particular, `QueryIterator{Ex}`
methods allow for the result of the internal query to be consumed in
"streaming" fashion but we must do so without any concurrency, and
it's achieved by using "sync" result channel (which synchronizes the
reader ("outer" query) and the writer (InternalExecutor)). However,
previously the internal query could result in illegal concurrency on its
own - if it happens to use DistSQL. During 23.1 time frame we started
populating the SessionData with proper session defaults in some cases
and completed this work in 23.2; as a result, the internally-executed
queries can now use DistSQL, but we must disable it in this "sync" mode,
when the root txn is specified, which is what this commit does.
We already made a similar change to disable the Streamer some time ago
for the same reason in ed3f640.
I spent quite a few hours to come up with a reproduction but didn't
succeed, so there is no regression test in this commit.
Release note (bug fix): Previously, CockroachDB could encounter an
internal error "attempting to append refresh spans after the tracked
timestamp has moved forward" in some cases when using virtual tables
(like `crdb_internal.system_jobs`, etc), and this has now been fixed.
The bug was introduced in 23.1 version.
This commit makes it so that the streamer is disabled in the internal executor in exactly the same scenario when DistSQL was disabled in the previous commit. This change polishes the fix in ed3f640 to be more precise. It's in a separate commit since it won't be backported (unlike the previous one). Release note: None
|
TFTR! bors r+ |
|
Build succeeded: |
This was referenced Feb 14, 2024
Closed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Previously, it was possible for some methods of the internal executor result in illegal concurrency. In particular,
QueryIterator{Ex}methods allow for the result of the internal query to be consumed in "streaming" fashion but we must do so without any concurrency, and it's achieved by using "sync" result channel (which synchronizes the reader ("outer" query) and the writer (InternalExecutor)). However, previously the internal query could result in illegal concurrency on its own - if it happens to use DistSQL. During 23.1 time frame we started populating the SessionData with proper session defaults in some cases and completed this work in 23.2; as a result, the internally-executed queries can now use DistSQL, but we must disable it in this "sync" mode, when the root txn is specified, which is what this commit does.We already made a similar change to disable the Streamer some time ago for the same reason in ed3f640.
I spent quite a few hours to come up with a reproduction but didn't succeed, so there is no regression test in this commit.
Fixes: #118542.
Release note (bug fix): Previously, CockroachDB could encounter an internal error "attempting to append refresh spans after the tracked timestamp has moved forward" in some cases when using virtual tables (like
crdb_internal.system_jobs, etc), and this has now been fixed. The bug was introduced in 23.1 version.