Switch branches/tags
Nothing to show
Find file
Fetching contributors…
Cannot retrieve contributors at this time
100 lines (71 sloc) 2.5 KB

Windex: Watch and Index

Windex is a library for concurrently watching log files and indexing their data.

A Watcher provides cross-platform file system notifications via These events help keep track of changes to the size of a file over time. This information is stored, and data from the tail of the file is sent via a channel to an external source, known as anIndexer.

The default Indexer flushes to stdout, effectively making windex a programatically accessible tail -f in its default state. However it can be much more powerful than that.

Indexer is an interface that contains two methods:

type Indexer interface {
	Parse(chan []byte) ([]byte, error)
	Flush([]byte) error

The stdout indexer does nothing with Parse(chan []byte), and simply prints with Flush([]byte). Since Parse has access to the data, it will be a powerful step toward implementing Indexers which can parse and flush data to any number of a variety of external sources: Redis, Postgres, Statsd, etc.

Here's an example naive implementation of a Redis Indexer:

type RedisIndexer struct {
	address string

func (i *RedisIndexer) Parse(log_data chan []byte) (parsed_log_data []byte, err error) {
	parsed_log_data = <-log_data
	return parsed_log_data, nil

func (i *RedisIndexer) Flush(parsed_log_data []byte) (err error) {
	redis, err := redis.Dial("tcp", i.address)

	if err != nil {

	redis.Do("lpush", "errorz", parsed_log_data)

	log.Print("Data Pushed: ", len(parsed_log_data), " bytes")


func NewRedisIndexer(address string) (stdout *RedisIndexer) {
	return &RedisIndexer{
		address: address,


// Use the windex generator to get a windex instance
windex, err = windex.New("logfile01.log")

// Launch the goroutine to watch our file
go windex.Watch()

// Launch the goroutine to index our data. The default indexer
// flushes data to stdout.
go windex.Index()

// Call select on our exit channel. This will let us keep
// everything moving and in the future may be of more use.
select {
case exit := <-windex.Exit:
	if exit {


The above is paraphrased from the example program in example/stdoutindexer/example.go. There is an example that uses the above Redis Indexer in example/redisindexer/example.go


  • Bring back the tests
  • Add wcld like Indexer


windex is (c) Michael R. Bernstein, 2012


windex is distributed under the MIT License, see LICENSE file for details.