Myloader hang fix, mydumper lost row fix, cleanups, tracing #1395
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
It's better to use correct quoting for DROP TABLE (for the case name contains backticks f.ex.).
MariaDB uses BACKUP STAGE STARTS for all versions >= 10.5. I don't see a reason why this should be disabled in later versions (f.ex. 11).
Human-readable thread names in trace instead of hex-based handles.
To cut-out timestamp from myloader trace:
cut_ts()
{
perl -pe 's/\d{4}-\d\d-\d\d \d\d:\d\d:\d\d \[\w+\] - //'
}
We have to send the exact quantity of DATA, we cannot shortcut them to just one DATA at INTERMEDIATE_ENDED. Now we pause control job thread by cond variable and resume it when the last S-thread receives JOB_SHUTDOWN.
Char-based chunking algorithm in mydumper is missing first row because of "<" in condition when comparing cmin_escaped. This comparison is wrong for the very first chunk and leads to first row missed. The fix includes first row in chunk and excludes last row unless last row is cmax. The problem was hidden in v0.15.2-6 because any chunking algorithm was disabled due to the bug fixed in 640ece1 (myloader ignores --max-threads-per-table). So v0.15.2-7 is affected by this bug since it enables chunking algorithms.
davidducos
approved these changes
Jan 26, 2024
|
@davidducos There is one more myloader hang known. The fix is undergoing. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Lost row fix requires hotfix release!
I think v0.15.2-7 was a bit premature. Lost row happens due to unmasked bug by 640ece1, so now v0.15.2-7 is spoiled.
There should be some stabilization period (a month) after so many changes before issuing a release.
Now as CircleCI blocked me it doesn't even run tests for my PRs. Tests passed locally.