correlating relay logs #32
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 3 commits
January 2, 2017 09:12
initial commit: using last known relay log coordinates to begin search for last relaylog entry
… destination servers
shlomi-noach
commented
Jan 2, 2017
| @@ -1261,7 +1261,7 @@ func FindLastPseudoGTIDEntry(instance *Instance, recordedInstanceRelayLogCoordin | |||
| } | |||
|
|
|||
| minBinlogCoordinates, minRelaylogCoordinates, err := GetHeuristiclyRecentCoordinatesForInstance(&instance.Key) | |||
| if instance.LogBinEnabled && instance.LogSlaveUpdatesEnabled && (expectedBinlogFormat == nil || instance.Binlog_format == *expectedBinlogFormat) { | |||
| if instance.LogBinEnabled && instance.LogSlaveUpdatesEnabled && !*config.RuntimeCLIFlags.SkipBinlogSearch && (expectedBinlogFormat == nil || instance.Binlog_format == *expectedBinlogFormat) { | |||
There was a problem hiding this comment.
This change is actually unrelated to this PR. However the need for it came at a good time and it makes sense to include it here.
added 3 commits
January 2, 2017 10:58
POC for AlignViaRelaylogCorrelation()
shlomi-noach
changed the title
shlomi-noach
changed the title
|
originally this PR was meant do go the whole way to relaylog-sync on failover. |
added 2 commits
January 5, 2017 08:28
…hestrator into failover-correlate-relay-logs
|
merging for now, continued work in other PRs |
This was referenced Jan 8, 2017
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Following up on #1, in the objective of aligning replicas at failover.
At this time this will cooperate with
orchestrator-agentat github/orchestrator-agent#13, though the same can be accomplished via remote SSH.When master fails,
orchestratoris able to use GTID/Pseudo-GTID to match replicas. However there are a few constraints: what if the most up-to-date replica doesn't have binlogging orlog-slave-updates? What if it uses aROWbased replication where all others useSTATEMENTbased? What if it's of a higher MySQL version?orchestratorwould have to lose it, even though it contained more data than others.This PR follows on the MHA approach, which requires either remote agents on MySQL boxes, or remote SSH. The intention is to correlate relay logs between failed replicas (done, optimized speed), then copy & apply such logs from the most up-to-date replica onto a (single) candidate replica.
Why single? Because the candidate replica would have
log-slave-updatesandorchestratorwould be able to point all other replicas under that one. It is yet to be seen whether comparing and copying relay logs onto a single replica, then applying Pseudo-GTID/GTID logic to heal the rest of the replicas, is faster or slower than comparing and copying relay logs from the most up-to-date replica onto all other replicas.Initial commits in this PR provide heuristic search for relay log coordinates & entries, which turn relay-log correlation into a subsecond operation, within
1minute from failure.