No description or website provided.
Python
Latest commit 8e250f3 Aug 18, 2015 @rhyolight rhyolight Merge pull request #5 from rhyolight/agpl
Updated to AGPL
Permalink
Failed to load latest commit information.
avogadro
docs
tools
.gitignore
LICENSE.txt
MANIFEST.in
README.rst
requirements.txt
setup.py

README.rst

Numenta Rogue

Metrics collection agent for the Numenta Rogue showcase application and consists of two primary components: A long-running metric collection agent, which periodically polls various metrics and records the results to a local database, and a separate process for forwarding metrics to a Grok server for analysis.

Each metric collection agent is executed in a continuously-running gevent Greenlet, with a pre-determined wait interval between samples. Each sample is cached locally in a dedicated RRDTool database. Individual metrics are implemented as subclasses of avogadro.agent.AvogadroAgent(), and must implement a collect() method, which returns the metric value.

The rogue-forward process fetches metric data from the individual RRDTool databases using the rrdtool fetch command, and sends the metric data using the Grok Custom Metric API. Each time the rogue-forward process is executed, the most recent timestamp is cached locally, and is used as a starting point for subsequent requests.

Usage

Install RRDTool:

brew install rrdtool

Install in development mode:

python setup.py develop --install-dir=... --script-dir=...

Start key-counter with nohup:

nohup rogue-keycounter > rogue-keys.stdout 2> rogue-keys.stderr < /dev/null &

Make sure that iTerm/Terminal is allowed in System Prefrences > Security & Privacy > Privacy > Accessibility!

Start metric collection agent with nohup:

nohup rogue-agent --prefix=var/db/ --interval=300 --hearbeat=600 > rogue-agent.stdout 2> rogue-agent.stderr < /dev/null &

Forward pending metric data to Grok once:

rogue-forward --server=https://localhost --prefix=var/db

Sample crontab entry:

* * * * * PATH=$PATH:/usr/local/bin PYTHONPATH=... .../rogue-forward --server=... --prefix=... > .../rogue-forward.stdout 2> .../rogue-forward.stderr < /dev/null

Export to local CSV

All metric data is written to a local round-robin database, which only retains the two most recent weeks of data at any given moment in time. Should you want the data exported to CSV, use the rogue-export utility:

rogue-export --prefix=var/db

The exporter keeps track of the position of a given metric, so you can run rogue-export as frequently as you like, and the .csv files in var/db/, will be updated accordingly. i.e. you can periodically sync the round-robin database to a csv file in var/db/

Metrics

CPUPercent

Total CPU utilization as a percentage, as reported by psutil.cpu_percent().

MemoryPercent

The percentage memory usage calculated as (total - available) / total * 100, as reported by psutil.virtual_memory().percent

DiskReadBytes

Number of bytes read (total), as reported by psutil.disk_io_counters.read_bytes

DiskWriteBytes

Number of bytes written (total), as reported by psutil.disk_io_counters.write_bytes

DiskReadTime

Time spent reading from disk (in milliseconds), as reported by psutil.disk_io_counters.read_time

DiskWriteTime

Time spent writing to disk (in milliseconds), as reported by psutil.disk_io_counters.write_time

NetworkBytesSent

Number of bytes sent, as reported by psutil.net_io_counters.bytes_sent()

NetworkBytesReceived

Number of bytes received, as reported by psutil.net_io_counters.bytes_recv()