Add --until-binlog-last-modified-time option to `wal-g-mysql binlog…
#1154
Conversation
…-replay` this may be useful to achieve exact clones of the same database in scenarios when new binlogs are uploaded during replay process of different hosts.
| @@ -24,7 +27,7 @@ var binlogFetchCmd = &cobra.Command{ | |||
| Run: func(cmd *cobra.Command, args []string) { | |||
| folder, err := internal.ConfigureFolder() | |||
| tracelog.ErrorLogger.FatalOnError(err) | |||
| mysql.HandleBinlogFetch(folder, fetchBackupName, fetchUntilTS) | |||
| mysql.HandleBinlogFetch(folder, fetchBackupName, fetchUntilTS, fetchUntilBinlogLastModifiedTS) | |||
There was a problem hiding this comment.
what's the difference between fetchUntilTS and fetchUntilBinlogLastModifiedTS ?
can we use fetchUntilTs instead of fetchUntilBinlogLastModifiedTS
There was a problem hiding this comment.
Here I am introducing new should distinguish two timestamps:
fetchUntilTS - is about time of individual transactions
fetchUntilBinlogLastModifiedTS - is about time when binlog was created
There are 2 properties:
fetchUntilTS < fetchUntilBinlogLastModifiedTS - because transactions dumped to binlog before binlog upload
--stop-datetime < fetchUntilBinlogLastModifiedTS - it is new rule that fixes multi-host restores (check detailed explanation in another comment)
| @@ -271,6 +271,9 @@ func getLogsCoveringInterval(folder storage.Folder, start time.Time, includeStar | |||
| }) | |||
| var logsToFetch []storage.Object | |||
| for _, logFile := range logFiles { | |||
| if logFile.GetLastModified().After(endBinlogTS) { | |||
| continue // don't fetch binlogs from future | |||
| } | |||
There was a problem hiding this comment.
This shold not affect the final result, as mysqbinlog started as
mysqlbinlog --stop-datetime="$WALG_MYSQL_BINLOG_END_TS"
and should not replay transactions happened after fetchUntilTS
There was a problem hiding this comment.
This patch fixes following issue:
Live MySQL cluster A constantly uploads binlogs. We are trying to restore new cluster (let's call it B) from S3 on some point in time, e.g. NOW() (or future time).
Then we can have following scenario:
'Fast' servers restore MySQL on Snapshot + binlog.123 (Binlog 123 is the latest observed binlog for 'fast' hosts).
'Slow' servers may restore MySQL on Snapshot+binlog.124 (Because cluster A have uploaded one more binlog during 'slow' restore).
As a result we have different GTIDs in a cluster. It is easier to prevent such scenario rather than fixing it afterwards.
The root cause of this issue is simple: newly uploaded binlog file may contain transactions that happend before user-specified --stop-datetime (it is arbitrary time, not a "$WALG_MYSQL_BINLOG_END_TS"). I can imagine scenario when binlog is uploaded once a day (after calling FLUSH BINARY LOGS). In this scenario for any user-specified timestamp from that day we will have transactions in new binlog.
To prevent different GTID-sets we should:
- point in time for this limits should be
NOW()or in past. - introduce hard limits on which binlogs we are replaying
This 2 rules guarantees that all restored clones will have same binlog files replayed and as a result - same GTID-sets.
Add
--until-binlog-last-modified-timeoption towal-g-mysql binlog-replaythis may be useful to achieve exact clones of the same database in scenarios when new binlogs are uploaded during replay process of different hosts.