Pre-configure telegraf's internal input plugin

[itzg]

Resolves

https://jira.rax.io/browse/SALUS-467

What

Telegraf provides an internal input plugin which provides self-monitoring metrics. That is a good one to auto-configure to provide some baseline data and liveness info about the telegraf agent even before the user has configured any monitors.

How

Since this changes (and simplifies) the "ready to run telegraf" condition, the logic has been tweaked to always ensure the main telegraf config file is written, the config.d directory is present, but does not need to contain any config files. The latter used to be a precondition since telegraf would error out if no input plugins were configured -- now we guarantee at least one input plugin is always configured.

I also changed the purge config logic slightly to ensure that a new main telegraf config file is written at least once per envoy startup. Otherwise, we didn't have a way to ensure the latest and greatest config file was re-written as we add features like this.

How to test

Existing unit tests and manually verified by inspecting the initial startup sequence:

time="2019-07-01T11:20:35-05:00" level=info msg="Using config file" file=envoy-config-gke-dev.yml
time="2019-07-01T11:20:35-05:00" level=debug msg="purging config" agentType=TELEGRAF
time="2019-07-01T11:20:35-05:00" level=debug msg="purging config" agentType=FILEBEAT
time="2019-07-01T11:20:35-05:00" level=debug msg="acquiring certificates from auth service" config="&{https://salus-auth-serv.dev.monplat.rackspace.net keystone_v2}"
time="2019-07-01T11:20:35-05:00" level=debug msg="acquiring keystone v2 authentication token" url="https://identity.api.rackspacecloud.com/v2.0/tokens"
time="2019-07-01T11:20:35-05:00" level=debug msg="acquired keystone v2 authentication token"
time="2019-07-01T11:20:36-05:00" level=info msg="successfully acquired certificates from auth service"
time="2019-07-01T11:20:36-05:00" level=debug msg="failed to get xen-id from cloud init" error="failed to read from instance id path: open /var/lib/cloud/data/instance-id: no such file or directory"
time="2019-07-01T11:20:36-05:00" level=debug msg="failed to get xen-id from xen client" error="no xen id found on os with type darwin"
time="2019-07-01T11:20:36-05:00" level=debug msg="unable to determine xen-id" error="no xen id found on os with type darwin"
time="2019-07-01T11:20:36-05:00" level=debug msg="unable to determine system serial number" error="no serial number found on os with type darwin"
time="2019-07-01T11:20:36-05:00" level=debug msg="unable to determine bios data" error="no bios data found on os with type darwin"
time="2019-07-01T11:20:36-05:00" level=debug msg="discovered labels" labels="map[discovered_arch:amd64 discovered_hostname:MS90HCG8WL discovered_os:darwin]"
time="2019-07-01T11:20:36-05:00" level=debug msg="Starting connection with identifier" resourceId="dev:geoff"
2019/07/01 11:20:36 Server config: server.options{timeout:30000000000, keepalive:3000000000, decoder:(server.jsonDecoder)(0x10f8c60), tls:(*tls.Config)(nil), v1:true, v2:true, ch:(chan *lj.Batch)(nil)}
2019/07/01 11:20:36 mk: 0
2019/07/01 11:20:36 mk: 1
time="2019-07-01T11:20:36-05:00" level=debug msg="Listening for lumberjack" address="localhost:5044"
time="2019-07-01T11:20:36-05:00" level=info msg="dialing ambassador" ambassadorAddress="salus-ambassador.dev.monplat.rackspace.net:443" envoyId=28c5d52a-9c1c-11e9-a058-c4b301c45f69
time="2019-07-01T11:20:36-05:00" level=info msg=attaching summary="supportedAgents:TELEGRAF supportedAgents:FILEBEAT labels:<key:\"discovered_arch\" value:\"amd64\" > labels:<key:\"discovered_hostname\" value:\"MS90HCG8WL\" > labels:<key:\"discovered_os\" value:\"darwin\" > resourceId:\"dev:geoff\" "
time="2019-07-01T11:20:37-05:00" level=info msg="processing install instruction" install="agent:<version:\"1.11.0\" > url:\"https://homebrew.bintray.com/bottles/telegraf-1.11.0.high_sierra.bottle.tar.gz\" exe:\"telegraf/1.11.0/bin/telegraf\" "
time="2019-07-01T11:20:37-05:00" level=debug msg="agent already installed" path=/Users/geof0549/git/salus-telemetry-bundle/dev/data-telemetry-envoy/agents/TELEGRAF/1.11.0 type=TELEGRAF version=1.11.0
time="2019-07-01T11:20:37-05:00" level=debug msg="ensuring telegraf is in correct running state"
time="2019-07-01T11:20:37-05:00" level=debug msg="creating main telegraf config file" path=data-telemetry-envoy/agents/TELEGRAF/telegraf.conf
time="2019-07-01T11:20:37-05:00" level=debug msg="starting agent" agentType=TELEGRAF cmd="&{CURRENT/bin/telegraf [CURRENT/bin/telegraf --config telegraf.conf --config-directory config.d] [] data-telemetry-envoy/agents/TELEGRAF <nil> 0xc0000b2088 0xc0000b2098 [] <nil> <nil> <nil> <nil> <nil> false [] [0xc0000b2088 0xc0000b2098] [0xc0000b2058 0xc0000b2090] [] <nil> <nil>}"
time="2019-07-01T11:20:37-05:00" level=info msg="2019-07-01T16:20:37Z I! Starting Telegraf 1.11.0\n" agentType=TELEGRAF
time="2019-07-01T11:20:37-05:00" level=info msg="2019-07-01T16:20:37Z I! Loaded inputs: internal\n" agentType=TELEGRAF