Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add heartbeat (experimental) #2743

merged 1 commit into from Oct 10, 2016


None yet
3 participants
Copy link

commented Oct 10, 2016

This PR introduces the new experimental heartbeat to the beats repository.

Heartbeat will monitor services for availability by active (and in future hopefully passive) probing. Heartbeat already supports ping via ICMP, TCP and HTTP. Including support for TLS. One can already configure some simple validators and even reconfigure single monitors without restarting heartbeat. Plus, given a hostname heartbeat can optionally ping all resolvable IPs.

See the default configuration file with all modules and settings available being documented.

Heartbeat provides a set of configurable monitor types somewhat similar to metricbeat modules, all supporting live (re-)loading most settings from external json based configuration files.

There are 3 kind of settings per module:

  • monitor manager settings. These are available for all monitors: type, enabled, watch.X
  • task generating settings:
    • icmp module uses hosts
    • tcp module uses hosts + ports creating tasks for all combinations
    • http module uses urls
  • task parameter settings: all settings regarding timeouts, validation + schedule.

Different kind of monitors might introduce different set of "task generating settings". The number of concurrent active I/O tasks can be limited using heartbeat.scheduler.limit.

All but the monitor manager settings can be overwritten using live (re-)loading. The watch namespace introduces configuration change watches in general with the type potentially being pluggable in the future. At the moment only poll_file is supported.

The file being watched consists of multiple JSON objects. By passing an empty file, all active monitoring tasks will be removed from the scheduler.

  • Every single JSON object in the file will be parsed and treated as separate configuration used to generate new tasks.
  • The objects are applied to the monitor in the same order as defined in the file being watched.
  • If any two objects create the same task (e.g. define same host:port), the most recent task being generated will be used.
  • Upon reload all active task will be removed first, before the new tasks will be started to deal with potentially parameter changes different from host name.
  • Applying an configuration objects means:
    • copy the original monitors settings
    • overwrite all settings in copy by settings found in json object
    • apply new configuration to monitor creating new, yet unscheduled tasks, potentially overwriting old tasks
  • Only after all configuration objects have been applied the monitoring tasks will be scheduled

Being able to overwrite literally any setting, also implies any monitor specific ping parameters can be overwritten/changed on the fly. E.g. timeouts for ICMP ping, certificates for TCP/TLS based ping, or validation settings for HTTP(S) based ping.

example: change TLS certificate_authorities for already configured host in monitor:

{"ssl.certificate_authorities": ["/path/to/new/certs"]}

example: add new TCP hosts with one adding a 'hello message' validator:

{"hosts": ["host1"]}
{"hosts": ["host2"], "ports": [80, 12345, 5044]}
{"hosts": ["host3:1234"], "check.receive": "hello world"}

@urso urso added the review label Oct 10, 2016

@urso urso force-pushed the urso:x/heartbeat branch from d46c26c to 3b151b8 Oct 10, 2016


This comment has been minimized.

Copy link

commented Oct 10, 2016

Great, 👍

@ruflin ruflin merged commit 614b05e into elastic:master Oct 10, 2016

4 checks passed

CLA Commit author is a member of Elasticsearch
continuous-integration/appveyor/pr AppVeyor build succeeded
continuous-integration/travis-ci/pr The Travis CI build passed
default Build finished.

This comment has been minimized.

Copy link

commented Oct 10, 2016

@urso Can you please open a meta issue in which we can track things that should still be done for heartbeat like docs etc?

@urso urso deleted the urso:x/heartbeat branch Feb 19, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.