Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

realtime log streamer

branch: master

Merge pull request #6 from mhorbul/master

Close all readers when main process got TERM signal
latest commit 0470169766
John Goulah jgoulah authored
Octocat-spinner-32 lib
Octocat-spinner-32 static Enhancements to the interactive Guide : June 30, 2012
Octocat-spinner-32 .gitignore
Octocat-spinner-32 README.md
Octocat-spinner-32 build first June 22, 2012
Octocat-spinner-32 localConfig.js first June 22, 2012
Octocat-spinner-32 package.json first June 22, 2012
Octocat-spinner-32 runlocal first June 22, 2012
Octocat-spinner-32 stream.js
Octocat-spinner-32 supergrep.init
README.md

Supergrep

Supergrep is a web based log streamer written in node. It can be used quite nicely to surface new log lines (errors, etc.) that aren't normally expected.

Essentially, having supergrep running in your browser while changes are being made allows for new/novel log patterns to show up, because under the hood, what we're really doing is:

$ tail -f {log filename} | grep -v {stuff you'd expect to see in log lines}

It's intended on being a noise reduction and change-awareness tool.

Prerequisites

In order to run supergrep, you need NodeJS and npm ( NodeJS package manager).

Getting Started

You can edit variables in localConfig.js and also in static/js/supergrep.js. Most of this should work out of the box but some of the cooler functionality is enabled by assuming certain pieces of data can be found in your logs. You'll have to adjust the regexes to suit your logs.

Then you can install the node packages:

$ npm install

Run it from your install directory like:

$ ./runlocal

Point your browser to localhost (default port:3000) and then click on "Guide" to get a tour.

Installation

After setting this up locally and seeing how it works you'll probably want to run in production. There are many init daemons that you can run this under. We've included a simple init.d script to get you started. This is known to work under RedHat flavers of linux. You will have to edit the OPTIONS to specify the path where supergrep is installed. It also sets a parameter for prodConfig.js that can take similar options as the localConfig.js that is included.

Log Format

Here is an example log line from our logs that this tool supports. You can modify it to support your log format.

web0081 : [Tue Aug 28 13:55:05 2012] [error] [client 10.101.136.5] [e4G2F4OpKIEiCj-eAQuEKo-H-XDB] [error] [ClientLogger] [/somepath/lib/Logger.php:140] [0] uncaught: message="Object doesn't support this action" referrer="http://www.etsy.com/search?q=juliana+bracelets&view_type=gallery&ship_to=ZZ&min=0&max=0&page=8" data="{"url":"http://lognormal.net/boomerang/cb5494ba3c140e877cba92969c4c9f8cd712d8af2f307f956895dd1d","line":5,"userAgent":"Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C)"}", referer: http://www.etsy.com/search?q=juliana+bracelets&view_type=gallery&ship_to=ZZ&min=0&max=0&page=9

Contributing

Patches welcome!

Something went wrong with that request. Please try again.