Statsdcc is a Statsd-compatible high performance multi-threaded network daemon written in C++. It aggregates stats and sends aggregates to backends.
Switch branches/tags
Nothing to show
Clone or download
Pull request Compare This branch is 24 commits ahead, 2 commits behind wayfair:master.
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
etc
include/statsdcc
lib
src
test
vendor
.gitignore
.gitmodules
CMakeLists.txt
LICENSE
README.md
makefile

README.md

statsdcc


Statsdcc is a Statsd-compatible high performance multi-threaded network daemon written in C++. It aggregates stats and sends aggregates to backends.

Dependencies


Compilation


git clone https://github.com/wayfair/statsdcc.git
cd statsdcc
git submodule init
git submodule update
mkdir build && cd build
cmake -D test=true ..
make
ctest

Usage


To run proxy

./src/proxy <config file>

To run aggregator

./src/statsdcc <config file>

Configuration


Example configuration files are in statsdcc/etc

Commmon Configuration Variables

  • servers: A list of server threads

    • udp: This variable is optional if tcp server is configured. To configure more than one threads to listen for metrics set variable threads. To override the size of udp recv buffer set recv_buffer variable. If recv_buffer not specified defaults to 8388608 bytes.

      "udp": {
      	"port": 9090,
      	"threads": 3,
      	"recv_buffer": 33554431
      }
      
    • tcp: This variable is optional if udp server is configured.

      "tcp": {
      	"port": 9090
      }
      
    • http: Http server listens for status check queries, by default uses port 8080.

      "http": {
      	"port": 8000
      }
      
  • workers: Number of worker threads to compute aggregations. If not specified defaults to 1.

  • log_level: Possible values "debug"/"info"/"warn"/"error". If not specified defaults to "warn". If set only logs with the specified level or higher will be logged.

Aggregator Configuration Variables

  • name: A custom name to the program. This name will be prefixed to status metrics generated by statsdcc. The default value is "statsdcc".

  • prefix: A value that will be appended to all metrics.

  • frequency: Interval in seconds to flush metrics to each backend. If not specified defaults to 10.

  • percentiles: For time metrics, calculate the Nth percentile(s). If not specified defaults to [90].

    "percentiles": [80, 90]
    
  • backends: List of backends to flush aggregations to. At least one of the following backends should be set.

    • stdout: Set to true to dump aggregations to stdout.

    • carbon: List of backend carbon instances.

      "carbon": [
      	{
      		"shard": "1",
      		"host": "localhost",
      		"port": 3101,
      		"vnodes": 1000,
      		"weight": 1
      	},
      	{
      		"shard": "2",
      		"host": "localhost",
      		"port": 3102,
      		"vnodes": 1000,
      		"weight": 1
      	}
      ]
      
      • shard: Key for this node used while building HashRing for consistent hasing.

      • host: Hostname for carbon instance. If not specified defaults to "127.0.0.1".

      • port: Tcp port number on which the carbon instance is listeningfor metrics. If not specified defaults to 3000.

      • vnodes: The amount of virtual nodes per server. Used for consistent hasing. Larger number gives bigger distribution in the HashRing. If not specified defaults to 1000.

      • weight: Similar to vnodes larger weight gives bigger distribution in the HashRing. If not specified defaults to 1.

  • repeaters: Using repeaters you can flush the aggregates to another statsdcc-aggregator or statsdcc-proxy.

    "repeaters": [
    	{
    		"host": "localhost",
    		"port": 8126
    	}
    ]
    
  • repeater_raw: When using a repeater should raw metrics be send instead of aggregates. Defaults to false.

    "repeater_raw": false
    

Proxy Configuration Variables

  • backends: List of backends to send metrics to, atleast one of the following backends should be set.

    • stdout: Set to true to dump aggregations to stdout.

    • aggregator: List of backend aggregator instances.

    "aggregator": [
    	{
    		"host": "localhost",
    		"port": 9090
    	},
    	{
    		"host": "localhost",
    		"port": 9091
    	}
    ]
    

Developer's References


  • UNIX Network Programming: W. Richard Stevens
  • Advanced Programming in the UNIX Environment: W. Richard Stevens, Stephen A. Rago
  • C++ Concurrency in Action: Practical Multithreading: Anthony Williams
  • Effective Modern C++: Scott Meyers