Implementation of the new binlog-server command

[Fizic]

Database name

mysql

Pull request description

wal-g runs MySQL implementation, and when a replica is connected to it, downloads binlogs from the repository and gives them over the replication protocol.

Currently requires the use of the library version provided in this pull-request: go-mysql-org/go-mysql#759

[lance6716]

Seems you can use go mod replace to let go-mysql use go-mysql-org/go-mysql#759 and let the CI work :)

[ostinru]

Wow! Great beginning!
I thought about this approach after reading ‘MySQL PiTR like a rockstar’[1][2][3] by @lefred
Below is my RFC I did some time ago, I hope you will find bits of it useful.

---

Proposal [stage 1]

add new command wal-g binlog-server that

• start listening a port
• parse requests on incoming connections (basic auth?)
• send all binlogs since backup timestamp (like we are doing it now) (known bug: MYSQL Recover failed: unknow database #1386)
• be able to stop sending events when reaching point in time

Open questions/discussion:

1. we should parse binlogs to find PiT. Is it fast to parse binlogs in golang? (anyway, mysqlbinlog also parses binlogs)
2. at first glance github.com/go-mysql-org/go-mysql lacks ready-to-use code for server message parsing

Proposal [stage 2]

This stage aims to make binlog-server generic: unlink it from backup_name.

for wal-g binlog-push - we should save GTID Sets to binlog sentinel, so binlog-server will be able to find the first binlog to serve

• Q: is there a chance for race between instances of wal-g writing the sentinel file?

for wal-g binlog-server we should be able to parse mysqld's COM_BINLOG_DUMP_GTID request (get GTIDSet from it) and find a set of binlog files to be sent to mysqld

Open questions/discussion:

1. Should we add ‘test’ command that will check that we have information about all required binlogs (to allow users to fallback to the old approach if we are not sure)

Other considerations

1. We should replay the first binlog from exact position: MYSQL Recover failed: unknow database #1386

---

Links

[1] https://lefred.be/content/howto-make-mysql-point-in-time-recovery-faster/
[2] https://www.slideshare.net/lefred.descamps/fosdem-mysql-friends-devroomffebruary-2018-mysql-pointintime-recovery-like-a-rockstar
[3] https://www.youtube.com/watch?v=PinKYCfv1MM&t=1787s