Skip to content
Simple OTP app for managing Erlang VM system_monitor event messages
Erlang Makefile
Latest commit df6e244 Oct 21, 2015 @javajolt javajolt Merge pull request #24 from basho/deps/jv/update-develop-dependencies
Update dependencies on develop branch
Failed to load latest commit information.
doc Add some extra usage/clarification Feb 3, 2011
ebin Bump app version to 2.0.0 Apr 11, 2014
priv Add runtime_health.triggers.process.long_schedule (based on the GC tr… Sep 30, 2014
src Address R17 namespaced_types. Feb 24, 2015
test
.gitignore Ignore PLT file. Feb 24, 2015
.travis.yml Updated travis hosts. Jul 10, 2015
LICENSE
Makefile Add tools.mk. Feb 24, 2015
README.md Add Travis CI hook and build status. Jun 27, 2012
rebar Update cuttlefish schema. Jan 13, 2014
rebar.config Update dependencies on develop branch Oct 21, 2015
tools.mk

README.md

riak_sysmon

Build Status

riak_sysmon is an Erlang/OTP application that manages the event messages that can be generated by the Erlang virtual machine's system_monitor BIF (Built-In Function). These messages can notify a central data-gathering process about the following events:

  • Processes that have their private heaps grow beyond a certain size.
  • Processes whose private heap garbage collection ops take too long
  • Ports that are busy, e.g., blocking file & socket I/O
  • Network distribution ports are busy, e.g., lots of communication with a slow peer Erlang node.

The problem with system_monitor events is that there isn't a mechanism within the Erlang virtual machine that limits the rate at which the events are generated. A busy VM can easily create many hundreds of these messages per second. Some kind of rate-limiting filter is required to avoid further overloading a system that may already be overloaded.

This app will use two processes for system_monitor message handling.

  1. A gen_server process to provide a rate-limiting filter.
  2. A gen_event server to allow flexible, user-defined functions to respond to system_monitor events that pass through the first stage filter.

There can be only one system_monitor process

(Silly reference to The Highlander omitted....)

The Erlang/OTP documentation is pretty clear on this point: only one process can receive system_monitor messages. But using the riak_sysmon OTP app, if multiple parties are interested in receiving system_monitor events, each party can add an event handler to the riak_sysmon_handler event handler.

The event handler process in this application uses the registered name riak_sysmon_handler. To add your handler, use something like: gen_event:add_sup_handler(riak_sysmon_handler, yourModuleName, YourInitialArgs).

See the gen_event documentation for add_sup_event/3 for API details. See the example event handler module in the source repository, src/riak_sysmon_example_handler.erl, for example usage.

Events sent to custom event handlers

The following events can be sent from the riak_sysmon filtering/rate-limiting process (a.k.a. riak_sysmon_filter) to the event handler process (a.k.a. riak_sysmon_handler).

  • {monitor, pid(), atom(), term()} ... These are system_monitor messages as they are received verbatim by the riak_sysmon_filter process. See the reference documentation for erlang:system_monitor/2 for details.
  • {suppressed, proc_events | port_events, Num::integer()} ... These messages inform your event handler that Num events of a certain type (proc_events or port_events) were suppressed in the last second (i.e. their arrival rate exceeded the configured rate limit).
Something went wrong with that request. Please try again.