Switch branches/tags
Find file History
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
..
Failed to load latest commit information.
README.md
main.go

README.md

Overview

Polymur-proxy is a daemon for off-site metrics forwarding over HTTPS. It connects to a remote Polymur-gateway instance with a valid API key issued by the gateway service. Mechanically, Polymur-proxy is a standard Polymur daemon with an HTTPS output writer. Therefore, Polymur-proxy can natively accept inputs from collectd, statsd and any other tools that implement the standard 'Graphite protocol'.

ScreenShot

Messages are batched, compressed (gzip; results in a ~5x reduction in outbound network bandwidth) and forwarded by a configurable number of workers (-workers directive) to the configured Polymur-gateway (-gateway directive).

Specifying a -cert is optional if using a self-signed certificate where it would otherwise fail as invalid.

Installation

Requires Go 1.6

  • go get -u github.com/jamiealquiza/polymur/...
  • go install github.com/jamiealquiza/polymur/cmd/polymur-proxy
  • Binary will be found at $GOPATH/bin/polymur-proxy

Usage

Polymur-proxy uses Envy to automatically accept all options as env vars (variables in brackets).

Usage of polymur-proxy:
  -api-key string
        polymur gateway API key [POLYMUR_PROXY_API_KEY]
  -cert string
        TLS Certificate [POLYMUR_PROXY_CERT]
  -console-out
        Dump output to console [POLYMUR_PROXY_CONSOLE_OUT]
  -gateway string
        polymur gateway address [POLYMUR_PROXY_GATEWAY]
  -listen-addr string
        Polymur-proxy listen address [POLYMUR_PROXY_LISTEN_ADDR] (default "0.0.0.0:2003")
  -metrics-flush int
        Graphite flush interval for runtime metrics (0 is disabled) [POLYMUR_PROXY_METRICS_FLUSH]
  -queue-cap int
        In-flight message queue capacity (number of data point batches [100 points max per batch]) [POLYMUR_PROXY_QUEUE_CAP] (default 32768)
  -stat-addr string
        runstats listen address [POLYMUR_PROXY_STAT_ADDR] (default "localhost:2020")
  -verbose
        Log verbosity [POLYMUR_PROXY_VERBOSE] (default true)
  -workers int
        HTTP output workers [POLYMUR_PROXY_WORKERS] (default 3)

Example Test Setup

Setup Polymur-gateway

See the Polymur-gateway readme

Run Polymur-proxy

$ ./polymur-proxy -cert="/path/to/cert.pem" -gateway="https://localhost:443" -api-key="test-key" -stat-addr="localhost:2021"
2016/07/29 14:26:53 ::: Polymur-proxy :::
2016/07/29 14:26:53 Pinging gateway https://localhost:443
2016/07/29 14:26:53 Connection to gateway https://localhost:443 successful
2016/07/29 14:26:53 HTTP writer #2 started
2016/07/29 14:26:53 HTTP writer #0 started
2016/07/29 14:26:53 HTTP writer #1 started
2016/07/29 14:26:53 Metrics listener started: 0.0.0.0:20032016/07/29 14:26:53
Runstats started: localhost:2020

Polymur-gateway authorizes key

2016/07/29 14:26:53 [client 127.0.0.1:50352] key for test-user is valid

Push Test Data

Send fake data point to Polymur-proxy

$ echo "some.test.data 1337 $(date +%s)" | nc localhost 2003

Polymur-proxy receives and forwards data

2016/07/29 14:30:54 [worker #2] sending batch (1 data points)
2016/07/29 14:30:54 [worker #2] [gateway] Batch Received
2016/07/29 14:30:58 Last 5.00s: Received 1 data points | Avg: 0.20/sec.

Polymur-gateway validates key and handles request (writes to console since the gateway was started with the -console-out flag)

2016/07/29 14:30:54 [client 127.0.0.1:50352] Recieved batch from from test-user
some.test.data 1337 1469802654
2016/07/29 14:30:55 Last 5.00s: Received 1 data points | Avg: 0.20/sec.