Ensure logTailer returns records in time order #6612

Merged
merged 6 commits into from Nov 25, 2016

Conversation

Projects
None yet
3 participants
Member

babbageclunk commented Nov 24, 2016

Having the separate sort by id worked by coincidence when records were
inserted in time order, but now that old records can be added by
migration the ids don't match up with times so records come back out of order.

Add id to the index instead and use one sort.

QA:
Deployed ubuntu to a new model
Noted the earliest log message for the model
Bootstrapped another controller
Migrated the model to the new controller
Checked that the earliest log message was the same

Member

babbageclunk commented Nov 24, 2016

Hmm, will this behave alright when there's an upgrade? I guess the first open post-upgrade will create the new index. Does there need to be something more explicit than this?

babbageclunk added some commits Nov 24, 2016

Ensure logTailer returns records in time order
Having the separate sort by id worked by coincidence when records were
inserted in time order, but now that old records can be added by
migration the ids don't match up with times. Add id to the index
instead and use one sort.
Add upgrade step to drop the old e,t logs index
state.InitDbLogs will create a new one on e,t,_id, but the old one needs
to be cleaned up at upgrade time.
Member

babbageclunk commented Nov 25, 2016

!!build!!

Member

babbageclunk commented Nov 25, 2016

Doing a manual test of the upgrade now.

Member

babbageclunk commented Nov 25, 2016

Upgrade worked fine.
Before:

juju:PRIMARY> db.logs.getIndexes()
[
        {
                "v" : 1,
                "key" : {
                        "_id" : 1
                },
                "name" : "_id_",
                "ns" : "logs.logs"
        },
        {
                "v" : 1,
                "key" : {
                        "e" : 1,
                        "t" : 1
                },
                "name" : "e_1_t_1",
                "ns" : "logs.logs"
        },
        {
                "v" : 1,
                "key" : {
                        "e" : 1,
                        "n" : 1
                },
                "name" : "e_1_n_1",
                "ns" : "logs.logs"
        }
]

After:

juju:PRIMARY> db.logs.getIndexes()
[
        {
                "v" : 1,
                "key" : {
                        "_id" : 1
                },
                "name" : "_id_",
                "ns" : "logs.logs"
        },
        {
                "v" : 1,
                "key" : {
                        "e" : 1,
                        "n" : 1
                },
                "name" : "e_1_n_1",
                "ns" : "logs.logs"
        },
        {
                "v" : 1,
                "key" : {
                        "e" : 1,
                        "t" : 1,
                        "_id" : 1
                },
                "name" : "e_1_t_1__id_1",
                "ns" : "logs.logs"
        }
]
Member

babbageclunk commented Nov 25, 2016

Gah. Not sure about these tests checking the values of constants.

Member

babbageclunk commented Nov 25, 2016

!!build!!

mjs approved these changes Nov 25, 2016

+ if err == nil {
+ return nil
+ }
+ if queryErr, ok := err.(*mgo.QueryError); ok {
@mjs

mjs Nov 25, 2016

Contributor

I was thinking you could check the output of Indexes() and then decide whether to drop, but this is fine too.

@babbageclunk

babbageclunk Nov 25, 2016

Member

I had that at first, but this seemed less racy. Although it does depend on looking at the error message.

babbageclunk added some commits Nov 25, 2016

Don't check the error code, apparently it changes
Manual testing showed that the upgrade step failed when the index wasn't
present, until the code check was removed.
Member

babbageclunk commented Nov 25, 2016

$$merge$$

Contributor

jujubot commented Nov 25, 2016

Status: merge request accepted. Url: http://juju-ci.vapour.ws:8080/job/github-merge-juju

@jujubot jujubot merged commit 25d8b1e into juju:develop Nov 25, 2016

@babbageclunk babbageclunk deleted the babbageclunk:debug-log-order branch Nov 25, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment