The purpose of the Bosh HM Forwarder is to forward bosh health metrics into Loggregator thus making them available in the firehose.
Bosh Health Metrics are currently emitted as bosh.healthmonitor.system.*
Currently the Bosh Director is capable of emitting metrics via its plugin architecture. The Bosh HM Forwarder acts as an OpenTSDB listener which reads off the metrics forwarded by the Bosh Health Monitor.
The Bosh HM Forwarder must be colocated with a fully functioning Metron Agent. It then forwards the metrics to Dopplers via the Metron Agent as ValueMetric type.
cd ~/workspace/loggregator/src/metron
- Use router_z1 on the cf_warden deployment to get the metron_agent configuration copy that information to /tmp/metron_config.json For Example:
{
"Index": 0,
"Job": "router_z1",
"Zone": "z1",
"Deployment": "cf-warden",
"EtcdUrls": ["http://10.244.0.42:4001"],
"EtcdMaxConcurrentRequests": 10,
"SharedSecret": "loggregator-secret",
"LegacyIncomingMessagesPort": 3456,
"DropsondeIncomingMessagesPort": 3457,
"EtcdQueryIntervalMilliseconds": 5000,
"LoggregatorDropsondePort": 3457,
"Syslog": "vcap.metron_agent"
}
go run main.go -config /tmp/metron_config.json
cd ~/workspace/loggregator/src/boshhmforwarder
- Create the /tmp/bosh-forwarder.json file
{ "IncomingPort":4001, "MetronPort": 3457 }
go run main.go --configPath /tmp/bosh-forwarder.json
-
cd ~/workspace/bosh-lite
-
vagrant ssh
-
sudo -i
-
vi /var/vcap/jobs/health_monitor/config/health_monitor.yml
-
add the following under 'plugins:' (line 32ish)
plugins: [...] - name: tsdb events: - alert - heartbeat options: host: <your HostIP> port: 4001
or (in case of JSON config)
"plugins": [{ "name": "tsdb", "events": ["alert", "heartbeat"], "options": { "host": "10.244.5.35", "port": 4000 } }, { "name": "logger", "events": ["alert"] }, { ... } ]
to find use
ifconfig
and look for:en0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500 ... inet 10.35.33.57 netmask 0xffffff00 broadcast 10.35.33.255 ... status: active
is the inet entry above like 10.35.33.57
-
monit restart health_monitor
- Add the loggregator release to the manifest
- name: loggregator
release: latest
- Add the boshhmforwarder to the instance group/job
- name: boshhmforwarder
release: loggregator
NOTE: Ideally, the Bosh HM Forwarder should be located on a single instance that has a Metron Agent on it. As such, it can be deployed on a separate vm with the Metron Agent colocated on it.