Skip to content


Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
A "tail -f" like file watcher for log indexing in Go
branch: master

Fetching latest commit…

Cannot retrieve the latest commit at this time

Failed to load latest commit information.

Windex: Watch and Index

Windex is a library for concurrently watching log files and indexing their data.

A Watcher provides cross-platform file system notifications via These events help keep track of changes to the size of a file over time. This information is stored, and data from the tail of the file is sent via a channel to an external source, known as anIndexer.

The default Indexer flushes to stdout, effectively making windex a programatically accessible tail -f in its default state. However it can be much more powerful than that.

Indexer is an interface that contains two methods:

type Indexer interface {
    Parse(chan []byte) ([]byte, error)
    Flush([]byte) error

The stdout indexer does nothing with Parse(chan []byte), and simply prints with Flush([]byte). Since Parse has access to the data, it will be a powerful step toward implementing Indexers which can parse and flush data to any number of a variety of external sources: Redis, Postgres, Statsd, etc.

Here's an example naive implementation of a Redis Indexer:

type RedisIndexer struct {
    address string

func (i *RedisIndexer) Parse(log_data chan []byte) (parsed_log_data []byte, err error) {
    parsed_log_data = <-log_data
    return parsed_log_data, nil

func (i *RedisIndexer) Flush(parsed_log_data []byte) (err error) {
    redis, err := redis.Dial("tcp", i.address)

    if err != nil {

    redis.Do("lpush", "errorz", parsed_log_data)

    log.Print("Data Pushed: ", len(parsed_log_data), " bytes")


func NewRedisIndexer(address string) (stdout *RedisIndexer) {
    return &RedisIndexer{
        address: address,


// Use the windex generator to get a windex instance
windex, err = windex.New("logfile01.log")

// Launch the goroutine to watch our file
go windex.Watch()

// Launch the goroutine to index our data. The default indexer
// flushes data to stdout.
go windex.Index()

// Call select on our exit channel. This will let us keep
// everything moving and in the future may be of more use.
select {
case exit := <-windex.Exit:
    if exit {


The above is paraphrased from the example program in example/stdoutindexer/example.go. There is an example that uses the above Redis Indexer in example/redisindexer/example.go


  • Bring back the tests
  • Add wcld like Indexer


windex is (c) Michael R. Bernstein, 2012


windex is distributed under the MIT License, see LICENSE file for details.

Something went wrong with that request. Please try again.