Fix two issues with binary backup (slow gzip compression and support for restore from instances other than SQL master) #551
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jianghao0718
approved these changes
Feb 11, 2020
jianghao0718
changed the title
jianghao0718
changed the title
This was referenced Feb 11, 2020
Closed
Merged
Merged
This was referenced Feb 11, 2020
Closed
Merged
Merged
dooleydevin
added a commit
that referenced
this pull request
Oct 2, 2023
…ne/find-parallel-in-more-locations Backport 527 for 3.8: Find parallel in more locations
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In this PR, we are trying to solve a few issues when backup and restore with binary backups
Use pigz instead of gzip in backup-util
Backup-util uses gzip which is super slow to compress large database files. It becomes the performance bottleneck when we backup binary backup. We will use pigz instead which is multi-threaded and way faster. (3 to 4 times faster for backing up large database)
Backup-util tool should always pointing to sql master for binary backup restore
This is an issue in current binary backup restore. Backup-util restores mysqldump from any instance in cluster because it connects to sql master through connection. Binary backup restore has different mechanism. We need to connect to the Sql master node and copy files there instead. Customer does not necessarily restore on the MySql Master node. (@lildude mentioned that we tell customers to restore from the MySQL master node, but seems the old logical backup restore works with non-master node so make this change to make sure it maintains same behavior)
Original PR: #548
@jianghao0718