Example: Scripts Pipeline

Pipeline Scripts

paStash can serve all sorts of script emitting JSON objects for distribution and post-processing. In this example, we'll use the input of ppsguard to power a gauge using the statsd protocol.

scola-pastash (colander)

input {
  udp {
    host => 127.0.0.1
    port => 1234
  }
}

filter {
  json_fields {}
}

output {
  studout {}
  statsd {
      host => localhost
      port => 8125
      metric_type => gauge
      metric_key => pps
  }
  influxdb {
      host => localhost
      port => 8089
      protocol => udp
      metric_type => counter
      metric_key => pps
  }
}

PPS Counter producer

npm install -g ppsguard
ppsguard --max_pps=100 --interface eth0 --udp 127.0.0.1:1234

Event

0|pastash  | [STDOUT] {
0|pastash  |   "cpu": 1.50927734375,
0|pastash  |   "pps": 101.996651754871111,
0|pastash  |   "mean": 141.236352751854155,
0|pastash  |   "message": "WARNING!"
0|pastash  |   "timestamp": "2018-01-02T17:33:44.737Z",
0|pastash  | }

---

PPS to Splunk using PM2

This example will produce PPS/CPU metrics using ppsguard and ship them to Splunk using pastash

Pastash

create new pastash pipeline config at /opt/pastash/ppsguard.conf

input {
  udp {
    host => 127.0.0.1
    port => 18911
  }
}

filter {
  json_fields {}
}

output {
  splunk {
    token => "XXX-YYY-ZZZ"
    source => "ppsGuard"
    index => "your-lovely-index"
    sourcetype => "_json"
    splunk_url => "https://your-own-id.splunkcloud.com:443/services/collector/event"
    flat => true
  }
}

PM2

create pm2 init file for ppsguard and pastash_pps in /opt/pastash/ppsguard_init.yml

apps:
 - script: ppsguard
   name: 'ppsguard'
   autorestart: true
   args: ['--interface', 'eth0', '--max_pps', '100', '--limit_every', 'second', '--udp', '127.0.0.1:18911']
 - script: pastash
   name: 'pastash_pps'
   autorestart: true
   args: ['--config_file=/opt/pastash/ppsguard.conf']

start the new services:

pm2 start /opt/pastash/ppsguard_init.yml

verify the service:

pm2 list