Simple daemon that consumes a syslog udp stream and generates statsd events when certain log lines are encountered.
statsdlog sample config:
[main] #debug = false # user to drop privs to #user = nobody #statsd_host = 127.0.0.1 #statsd_port = 8125 #statsd_sample_rate = 1.0 #listen_addr = 127.0.0.1 #listen_port = 8126 #buffer_size = 8192 #max_line_backlog = 512 #report internals stats about statsdlog to statsd #report_internal_stats = false # if enabled report internal stats every 5 seconds #internal_stats_interval = 5 # file that contains your log search patterns and statsd event names patterns_file = /etc/statsdlog/patterns.json # json_pattern_file = true # # alternatively instead of using a json based patterns file you can use a # a plain text format: # patterns_file = /etc/statsdlog/patterns.conf # json_pattern_file = false
- Copy etc/statsdlog/patterns.json to /etc/statsdlog/patterns.json
- Edit patterns.json to include the regex patterns for log lines you want to fire events for.
- Theres a few examples included in etc/statsdlog/ (everything from firing events when packages are installed to nginx errors)
- Copy etc/statsdlog/statsdlog.conf-sample to /etc/statsdlog/statsdlog.conf
- Edit the the conf file to point to your statsd host
- Point the relevent syslog udp stream to 127.0.0.1:8126
sudo python bin/statsdlog-server --conf=/etc/statsdlog/statsdlog.conf start(or use the the included init script)
Its important to note that the first match NO LONGER wins. As of v0.1.1 an event will be fired for EACH line match. This means a single log line can generate multiple events.
The included patterns.json-openstackswift example includes a few patterns for errors commonly encountered when running swift. There are also sample patterns for things like Nginx errors, sudo/ssh failures, or package installs. statsdlog is handy for generating events for all the things you want to track that do not natively support statsd.
git clone git://github.com/pandemicsyn/statsdlog.git
python setup.py install
etc/statsdlog/statsdlog.init is available as a simple init script. You may need to update it to point to /usr/local/bin/ if installing via pip or setup.py.
pip install statsdlog
Clone the version you want and build the package with stdeb (sudo apt-get install python-stdeb):
git clone firstname.lastname@example.org:pandemicsyn/statsdlog.git statsdlog cd statsdlog python setup.py --command-packages=stdeb.command bdist_deb dpkg -i deb_dist/python-statsdlog_0.0.6-1_all.deb