Implement server stats and self-monitoring

[otoolep]

No description provided.

[otoolep]

$ curl -G http://localhost:8086/query?pretty=true --data-urlencode "q=SHOW STATS"
{
    "results": [
        {
            "series": [
                {
                    "name": "server",
                    "columns": [
                        "batchWriteRx",
                        "broadcastMessageRx",
                        "broadcastMessageTx",
                        "pointWriteRx",
                        "queryRx",
                        "queryStatementRx",
                        "rowsReturned",
                        "shardWriteByte",
                        "shardWriteTime",
                        "shardsCreated",
                        "writeSeriesMessageRx",
                        "writeSeriesMessageTx"
                    ],
                    "values": [
                        [
                            15
                        ],
                        [
                            74
                        ],
                        [
                            64
                        ],
                        [
                            47
                        ],
                        [
                            78
                        ],
                        [
                            78
                        ],
                        [
                            40
                        ],
                        [
                            1350
                        ],
                        [
                            468990206
                        ],
                        [
                            9
                        ],
                        [
                            15
                        ],
                        [
                            15
                        ]
                    ]
                }
            ]
        }
    ]
}

[mark-rushakoff]

Missing an interval := config.Statistics.WriteInterval?

[otoolep]

Very good -- I just fixed that on my local copy and see you spotted it. Fix coming.

[otoolep]

There is 1 piece remaining for this, the ability to query a different server. This is being tracked by #1945

[otoolep]

Green build.

[mark-rushakoff]

Since the self-monitoring is a form of instrumentation, we should probably be using a time.Ticker.

For a good demonstration of the difference between the two, compare the output of using time.Sleep vs. using time.NewTicker.

[otoolep]

Ah, yes, I know about the tickers. I agree it would be nice to be precise with this stuff.

[otoolep]

Updated to use time.Ticker.

[toddboom]

@otoolep Would you mind rebasing this? If you think it's ready to go, I'll merge it in for RC12.

[otoolep]

I have rebased this and the build is green. I have opened a ticket for the unrelated Travis failure. #1962

I consider this code ready to merge, though there is still significant more work to do in terms of the stats we need to collect. We also "lost" some stats due to the Raft refactor, since some stats are now only available at a lower-level than the server object. I will need to take a look at how to gather those important stats within the new framework.

This code has not been reviewed, so please review before merging.

[toddboom]

Awesome, this looks good to me. I'll give everyone else a chance to review and then I'll merge in a couple hours.