Logsandra
=========
Logsandra is a log management application written in Python and using Cassandra as back-end.


Requirements
------------
Cassandra 0.6.x, Python 2.6/2.7 (not tested with Python 3.x) and the following Python modules:

    * Pylons >= 1.0
    * Jinja2
    * PyYAML
    * Pycassa
    * python-dateutil
    * CherryPy >= 3.1
    * Thrift
    * ordereddict
    * Cassandra (not automatically installed during the installation phase)


Installation
------------
To install Logsandra you run this command:

    python setup.py install

All Python modules will be automatically installed by that command except for the Cassandra bindings.


Configuration
-------------
To use Logsandra, you need to configure your Cassandra cluster. Below is the current Keyspace configuration needed by Logsandra:

    <Keyspaces>
        <Keyspace Name="logsandra">
            <ColumnFamily Name="entries" CompareWith="BytesType" />
            <ColumnFamily Name="by_date" CompareWith="LongType" />
            <ColumnFamily Name="by_date_data" CompareWith="LongType" />
            <ColumnFamily Name="categories" CompareWith="UTF8Type" />
          <ReplicaPlacementStrategy>org.apache.cassandra.locator.RackUnawareStrategy</ReplicaPlacementStrategy>
          <ReplicationFactor>3</ReplicationFactor>
          <EndPointSnitch>org.apache.cassandra.locator.EndPointSnitch</EndPointSnitch>
        </Keyspace>
    </Keyspaces>

(In Cassandra 0.7 and higher you will not need to change the Keyspace configuration manually, but as for now Cassandra 0.7 is in development and not supported by Logsandra)


The main configuration file for Logsandra is logsandra.yaml, the format is yaml. Below is a list of configuration variables (including type and optional/required):

    ident               identity of the logsandra server [String, Optional/Discovery]

    httpd_host          host on which to serve the logsandra web service [String, Required]
    httpd_port          port on which to serve the logsandra web service [String, Required]
    httpd_config        name of the pylons configuration file [String]

    cassandra_host      cassandra host to connect too [String, Required]
    cassandra_port      cassandra port to connect too [String, Required]
    cassandra_timeout   in seconds until connection is aborted if no response is delivered [String, Optional]

    paths               list of paths to monitor [List, Optional]
      path              directory or file [String, Required]
      recursive         recursively enter sub directories [True/False, Optional]
      parser            dictionary containing parser name and arguments [Dictionary, Required]
        name            name of the parser [String, Required]
        ...             arguments, see specific parser [Depends, Optional]

Logsandra is built on Pylons and therefore a Pylons configuration file should be present for Logsandras web service to work.


Usage
-----
Logsandra consists of two main scripts, which both acts as a daemon, below is the usage pattern:

    python logsandra-httpd.py [options] start|stop|restart
    python logsandra-monitord.py [options] start|stop|restart

If you want a list of options invoke either one these commands:

    python logsandra-httpd.py --help
    python logsandra-monitord.py --help

An example setup is to run logsandra-httpd on one server and logsandra-monitord on all servers producing log files. Logsandra-monitord will send the data to the specified Cassandra Cluster. 


Create your own parser
----------------------
To create your own Logsandra parser you need to create a python file in this directory:

    logsandra/monitor/parsers/

The name of the file should be in lower case and end with '.py', an example of a files content (the filename is foo.py):

    from logsandra.monitor.parsers import BaseParser

    class FooParser(BaseParser):
        def parse(self, line, source, data):
            date = None
            keywords = []
            return self.log_entries.add(date=date, entry=line, source=source, keywords=keywords)

A better example is to study how existing parsers work in logsandra/monitor/parsers/ directory.