Shh - Systems Heuristics Herald
Go Makefile
Clone or download
Pull request Compare This branch is 2 commits behind heroku:master.
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
cmd
misc
vendor
.gitignore
.travis.yml
CHANGELOG.md
Dockerfile
LICENSE
LISTEN.md
Makefile
POLLERS.md
Procfile
README.md
ato_test.go
carbon_output.go
cgroup_poller.go
config.go
conntrack_poller.go
cpu_poller.go
df_poller.go
disk_poller.go
fields_test.go
file_test.go
filenr_poller.go
folsom_poller.go
folsom_poller_test.go
get_env_with_test.go
librato_output.go
librato_output_test.go
listen_poller.go
listen_poller_test.go
load_poller.go
log.go
mem_poller.go
misc_test.go
mm.go
nagios3stats_poller.go
nif_poller.go
ntpdate_poller.go
outputters.go
pollers.go
processes_poller.go
redis_poller.go
self_poller.go
shh_test.go
sockstat_poller.go
splunk_search_peers_poller.go
statsd_output.go
stdout_output.go
string_slice_test.go
syslog-ng_stats.go
units.go
utils.go

README.md

Travis Releases GoDoc

System Heuristics Herald (aka Shh)

Gathers and relays system metrics

Install

go get github.com/heroku/shh

Configuration

Configuration of shh doesn't use a config file, instead it uses environment variables.

Environment Var Type Explanation Default
SHH_INTERVAL duration Polling Interval 10s
SHH_META bool Report/Collect meta stats false
SHH_OUTPUTTER string Outputter stdoutl2metder
SHH_POLLERS list of string Pollers to poll conntrack,cpu,df,disk,listen,load,mem,nif,ntpdate,processes,self
SHH_SOURCE string Source to emit
SHH_PREFIX string Metric prefix to use
SHH_PROFILE_PORT string Profile Port 0 (off)
SHH_PERCENTAGES list of string Default pollers which should report percentages when applicable
SHH_DF_TYPES list of string Default DF types btrfs,ext3,ext4,tmpfs,xfs
SHH_LISTEN string Default network socket info for listen unix,#shh
SHH_LISTEN_TIMEOUT string Socket timeout duration SHH_INTERVAL
SHH_NIF_DEVICES list of string Devices to poll eth0
SHH_NTPDATE_SERVERS list of string NTP Servers 0.pool.ntp.org,1.pool.ntp.org
SHH_CPU_AGGR bool Whether to only report aggregate CPU usage true
SHH_LIBRATO_USER string The Librato API User
SHH_LIBRATO_TOKEN string The Librato API Token
SHH_LIBRATO_URL string The Librato API User https://metrics-api.librato.com/v1/metrics
SHH_LIBRATO_BATCH_SIZE int The max number of metrics to submit in a single request 500
SHH_LIBRATO_BATCH_TIMEOUT duration The max time metrics will sit un-delivered SHH_INTERVAL
SHH_LIBRATO_ROUND bool Should shh round times to the nearest interval? true
SHH_NETWORK_TIMEOUT duration Timeout til connect (will retry). And timeout to first header (will assume successful). Used for HTTP(S) endpoints and other network communication 5s
SHH_CARBON_HOST string Where the Carbon Outputter sends it's data
SHH_SOCKSTAT_PROTOS list of string Protocols to report sockstats about TCP,UDP,TCP6,UDP6
SHH_STATSD_HOST string Where the Statsd Outputter sends it's data
SHH_STATSD_PROTO string Whether the Stats Outputter uses TCP or UDP udp
SHH_SYSLOGNG_SOCKET string The location of the syslog-ng socket /var/lib/syslog-ng/syslog-ng.ctl
`SHH_FULL list of strings Pollers that should report full metrics. shh defaults to minimal ""
SHH_DISK_FILTER regexp Scan devices that match this regex (xv
SHH_PROCESSES_REGEX regexp Scan / extract metrics for processes that match this regex \A\z
SHH_TICKS int cpu ticks per second: see getconf CLK_TCK. Default is probably correct. (temporary until we use cgo) 100
SHH_PAGE_SIZE int system page size in bytes: see getconf PAGESIZE. Default is probably correct. (temporary until we use cgo) 4096
SHH_NAGIOS3_METRIC_NAMES list of strings list of nagios 3 metric names to report stats on, see nagios3stats -h NUMSERVICES,NUMHOSTS,AVGACTSVCLAT,AVGACTHSTLAT,NUMHSTACTCHK5M,NUMSVCACTCHK5M,NUMHSTACTCHK1M,NUMSVCACTCHK1M
SHH_SPLUNK_PEERS_SKIP_VERIFY bool whether or not to skip verification of HTTPS cert on splunk peers endpoint false
SHH_SPLUNK_PEERS_URL string URL of splunk distributed peers status (e.g. https://user:pass@localhost:8089/services/search/distributed/peers?count=-1
SHH_FOLSOM_BASE_URL string URL of exported folsom metrics via folsome_cowboy or folsom_webmachine (e.g. https://localhost:5564/)
SHH_REDIS_URL string URL for Redis as defined by goredis (e.g. tcp://auth:password@127.0.0.1:6379/0?timeout=10s&maxidle=1) tcp://localhost:6379/0?timeout=10s&maxidle=1
SHH_REDIS_INFO string Description of INFO: section0:key0,key1;section1:key0,key1 to pull clients:connected_clients;memory:used_memory,used_memory_rss;stats:instantaneous_ops_per_sec;keyspace:db0.keys
SHH_CGROUPS list of string cgroups to report stats on group1,group2,group3

For more information on the duration type, see time.ParseDuration

The regexp type supports valid regexps documented here.

A note about SHH_OUTPUTTER

The SHH_OUTPUTTER variable may not be enough on it's own to get the desired result. For instance, the Librato outputter, requires that SHH_LIBRATO_USER and SHH_LIBRATO_TOKEN be set.

A note about SHH_PERCENTAGES

This variable works on "virtual" pollers and computes "percentage used", reporting as ".perc"

  • mem (from the mem poller)
  • swap (from the mem poller)
  • df (from the df_poller)

Building Debs

Requirements:

  • dpkg (see also brew install dpkg)
  • go & gox, which is installed via the Makefile
make debs

Note: You can find debs on the Github release page

'Local' Development

  1. Obtain a Linux system (only really tested on Ubuntu ATM)
  2. Install Go (version 1.4+)
  3. Set GOPATH appropriately
  4. go get github.com/tools/godep
  5. go get github.com/heroku/shh
  6. cd $GOPATH/src/github.com/heroku/shh
  7. go test -v ./...

hack away

Contributing

The goal for shh is to be a stable, low footprint system metrics poller, and we welcome contributions, feedback and bug reports to make that happen.

We're currently focused on supporting GNU/Linux systems, since that's where we're using shh, but are open to supporting other platforms provided the low footprint nature is preserved.

Please file bug reports through Github Issues. If you'd like to contribute changes, please fork and submit a pull request.

License

Copyright 2013 - 2015, Edward Muller, and contributors

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program. If not, see http://www.gnu.org/licenses/.