Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding reconnection on mydumper #1409

Merged
merged 4 commits into from
Feb 13, 2024
Merged

Adding reconnection on mydumper #1409

merged 4 commits into from
Feb 13, 2024

Conversation

davidducos
Copy link
Member

@davidducos davidducos commented Feb 12, 2024

I created a table with 5M rows without secondary indexes:

sysbench /usr/share/sysbench/oltp_write_only.lua --table-size=5000000 --tables=1 --threads=1 --mysql-user=root --mysql-db=sbtest --rate=10 --time=0 --report-interval=10 --create_secondary=off prepare

to kill just one thread, I created this:

echo "select concat('Kill ',ID) from information_schema.PROCESSLIST where INFO like 'SELECT %sbtest%' and ID != connection_id() limit 1" | mysql -N | mysql

The mydumper command was tested with GTID ON and OFF and this 2 combinations:

rm -rf data/; ./mydumper -B sbtest -o data -v 3 -M -r 1000000 -t 2 -F 20 --no-locks
rm -rf data/; ./mydumper -B sbtest -o data -v 3 -M -r 1000000 -t 2 -F 20 

There are multiples combinations and considerations, the most important is that reconnection is not possible during what mydumper considers CONSISTENT backup. For instance, if you use --no-locks on a stopped replica, over MySQL, mydumper will consider it is inconsistent, even if the database is not receiving traffic, but if you do the same on Percona with GTID ON, it will be a consistent backup.

@davidducos davidducos added this to the Release 0.15.1-3 milestone Feb 12, 2024
…rror messages, and adding consistency checks as reconnection might cause inconsistent backups
@davidducos davidducos merged commit fa163a4 into master Feb 13, 2024
32 of 34 checks passed
@davidducos davidducos deleted the enh branch March 12, 2024 15:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant