Feature Request: MySQL Shell Logical Backups

[rvrangel]

Feature Description

Currently Vitess only supports 2 backups engines, both of which take physical backups. Adding support for logical backups generated by MySQL Shell would complement other scenarions that require this type of backup.

The main interest in MySQL Shell dumps over mysqldump is because how much faster they are. You can see some details in these benchmarks on the MySQL Blog, and that is the main reason of interest over potentially really long mysqldump backups.

mysqldump would actually fit the vitess backup flow a lot better since it generates a stream of data that can easily be compressed and stored on its respective storage engine. MySQL Shell on the other hand, stores the output on a directory, or alternatively can directly upload to different object storages that it supports.

This means that support in Vitess will have to be a bit different. The initial idea was that we could pass the right flags so that MySQL Shell nows where to store the backup (either locally or to a remote location) and that we save this information on the MANIFEST file. This way, we would only have the metadata itself being handled by Vitess, while the backupd data would be handled "out of band" by MySQL Shell

There are some other challenges also when restoring because during backup time, we can't capture the GTID position of when the backup is stored (although that gets stored with the backup, so MySQL Shell is aware of it when doing the restore). A way around that is keeping it empty on the manifest file, and during the restore, between MySQL Shell completing the restore and us handing it over backup to Vitess, we can read the position from MySQL and override it in the manifest, effectively completing the full backup -> restore cycle.

I would be interested to see what is the interest in supporting this additional use case, as it differs a bit of how the other 2 backup engines work.

Use Case(s)

Logical backups can be used to recover in situations where a potentially corrupted physical backup would cause a lot more trouble to restore from and is also a requirement in some cases for compliance reasons.