Herald is a load feedback and check agent for Haproxy.
The agent is configured in Haproxy using the
agent-check server option. Once enabled haproxy periodically (
agent-inter) connects to the backend on the configured port (
agent-port) and acts on the response.
The response can either indicate an action to perform, such as
DRAIN, etc or change the weight of the respective backend with a percentage, say
75%. Check Haproxy documentation here for more details.
There are many use cases for using the agent, load feedback being an obvious one.
- Flexible and easy to configure
- Extensible using plugins
- High performance
We recommend using pip to install in a virtualenv :
$ virtualenv herald $ pip install haproxy-herald
In your haproxy backend add the agent-check configuration like so :
frontend myservice bind *:8004 maxconn 20000 option tcp-smart-accept default_backend myservers backend myservers balance leastconn option tcp-smart-connect fullconn 20000 server myserver01 myserver01:8081 maxconn 10000 weight 100 check agent-check agent-port 5555 server myserver02 myserver02:8081 maxconn 10000 weight 100 check agent-check agent-port 5555 server myserver03 myserver03:8081 maxconn 10000 weight 100 check agent-check agent-port 5555
This instructs haproxy to connect to agent on port 5555.
On the backend servers, run herald with the following configuration :
--- name: myservice bind: 0.0.0.0 port: 5555 plugins_dir: /etc/herald/plugins plugins: - name: myservice_plugin herald_plugin_name: herald_http url: 'http://localhost:8081/health_check' is_json: yes stop_timeout: 60 interval: 30 staleness_interval: 120 staleness_response: noop thresholds_metric: "r['msg-pre-second']" thresholds: - pct: 7000 min_threshold_response: 1 default_response: noop
Herald reads from
/etc/herald/config.yml by default. The plugins are bundled in herald.plugins, which can be symlinked in
Check the example_config.yml file for detailed configuration options.
With this configuration, herald will poll the health check url every 30s. Note that the response is also cached to avoid hitting the health check url too often.
The herald_http and herald_file plugins are provided in herald/plugins directory. These should serve most use cases. The plugin code is simple and easy to follow; Writing additional plugins should be very easy.
The following features are provided by the plugin framework :
- Check result cacheing
- Json parsing and processing parsed data using any python dict expression
- Arthmetic expressions on the result
- Calculate weight percentage on the result
- Regex pattern matching on the result
- Unit and integration tests
- Improved logging
- Configuration reload on SIGHUP