Add overlay metrics.

[ichernetsky]

High-level description

Add overlay metrics to ease detection of common issues users of the overlay module encounter.

Changelog automation

DCOS_OSS-5321 Add overlay metrics.

[ichernetsky]

Team, could you please let me know if these metrics names are fine with you. What do you think of having overlay/master and overlay/slave prefixes? If you think this distinction makes sense, what do you think about renaming overlay/log/ensemble_size and overlay/log/recovered to overlay/master/log/ensemble_size and overlay/master/log/recovered respectively? The Telegraf Mesos plugin doesn't seem to collect them as of now.

[urbanserj]

@ichernetsky I really like your idea! btw how do we convert mesos metric names, i.e. what names will be in prometheus?

[ichernetsky]

@urbanserj, metrics names are used as is in the plugin itself, and AFAIU, the Telegraf's Prometheus output plugin replaces invalid (from the POV of Prometheus) characters with underscores. Please refer to https://github.com/dcos/telegraf/blob/1.9.4-dcos/plugins/inputs/mesos/mesos.go for further details.

[ichernetsky]

The Telegraf plugin adjustments are here: dcos/telegraf#59
This one should (obviously) be merged before the PR against Telegraf repo.

[ichernetsky]

@GoelDeepak, do I have your blessing to merge it?