migrations: track the latest log record seen in logtransfer #6641

Merged
merged 4 commits into from Dec 1, 2016

Conversation

Projects
None yet
4 participants
Member

babbageclunk commented Nov 30, 2016

The migration logtransfer endpoint now keeps track of the latest time it's seen to enable restarting the transfer if it's interrupted. The migrationtarget facade has a LatestLogTime method so the migrationmaster can specify the start time to the debug log endpoint.

The latest time is updated every two minutes of log records seen (so as not to spam the database), and when the connection is closed. This means that the log time will be up-to-date unless the target controller is killed for some reason.

Member

babbageclunk commented Nov 30, 2016

!!build!!

LGTM. I am not familiar with the migration stuff. So you might want another set of eyes.

@@ -127,6 +128,9 @@ func (c *Client) httpPost(modelUUID string, content io.ReadSeeker, endpoint, con
return nil
}
+// OpenLogTransferStream connects to the migration logtransfer
+// endpoint on the target controller and returns a stream that JSON
+// logs records can be fed into. The objects written should be params.LogRecords.
@reedobrien

reedobrien Nov 30, 2016

Contributor

Thanks for adding docs:)

@babbageclunk

babbageclunk Nov 30, 2016

Member

:) That was a method I added in an earlier PR, but I realised I hadn't documented it.

Member

babbageclunk commented Nov 30, 2016

!!build!!

babbageclunk added some commits Nov 30, 2016

Track the last log time in migration-logtransfer
Write to the database once every two minutes of log time, and when the
connection closes. This will be used by the migration master to
determine where to start when streaming logs from the debug-log endpoint.
Add migrationtarget.LatestLogTime
This returns the time of the most recent log message seen by the
logtransfer endpoint, so that it can be used as the start time for the
log source when restarting.

mjs approved these changes Dec 1, 2016

apiserver/logtransfer.go
+ return &logTracker{tracker: state.NewLastSentLogTracker(st, st.ModelUUID(), "migration-logtransfer")}
+}
+
+type logTracker struct {
@mjs

mjs Dec 1, 2016

Contributor

The tracker only works correctly as long as the logs are seen in order right?

I realise that this is probably the case but it might be worth documenting that around here.

@babbageclunk

babbageclunk Dec 1, 2016

Member

Yeah - I could make it a high-water thing but that seems like overkill for this.
I'll mention that assumption in various places.

Member

babbageclunk commented Dec 1, 2016

$$merge$$

Contributor

jujubot commented Dec 1, 2016

Status: merge request accepted. Url: http://juju-ci.vapour.ws:8080/job/github-merge-juju

@jujubot jujubot merged commit 7f63dd6 into juju:develop Dec 1, 2016

@babbageclunk babbageclunk deleted the babbageclunk:logtransfer-track branch Dec 1, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment