Skip to content
Django application to store log messages on Cassandra
Python JavaScript Shell
Find file
New pull request
Fetching latest commit...
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.


Daedalus is a Django application to store log messages on Cassandra. The messages are sent/received using HTTP POST, encoded as a JSON dictionary.

There's a basic wiki at github.

This project is al alpha-quality stage, not recommended to be used on production systems.

It's developed on Ubuntu 12.04, and tested on CentOS 6 and Ubuntu 12.04 LTS Server virtual machines (with the help of fabric).

Click here to report any issue.

Implemented functional use cases

  1. Backend: Receive log messages using HTTP (POST with JSON encoded message)

  2. Frontend: Show messages

    • Filter by application
    • Filter by host
    • Filter by severity
    • Show all messages (default for home page)
    • Simplest form of pagination
    • Show line chart counting messages received
  3. Client: Python client to send messages using HTTP (POST with JSON encoded message)

For the curious: install in a virtual machine

See dev-scripts/

I recommend you to download and run the script in a newly created virtual machine (mainly because it install many packages, and must be run as root). The virtual machine should have at least 1GB of RAM (Cassandra may not work with less memory). The scripts installs JDK, Cassandra, clones the Daedalus repository and launch the Django development server.

You can download this script and run it as root, or use it as a guide, copying-and-pasting each of the commans of the script in a console or ssh session.

For developers: how to download and hack

JDK: download and install JDK 6.

Cassandra: download, install and start Cassandra.

Download from GitHub

$ git clone
$ cd daedalus

Create virtualenv and install requeriments

$ virtualenv virtualenv
$ ./virtualenv/bin/pip install -r requirements.txt
$ ./virtualenv/bin/pip install -r requirements-dev.txt

Run syncdb and syncdb_cassandra

$ ./dev-scripts/ syncdb
$ ./dev-scripts/ syncdb_cassandra

Start memchaed

$ sudo service memcached start

Start development server

$ ./dev-scripts/

By now you'll have the Django development server running. Both the Daedalus backend (the Django app that receives the logs via HTTP) and the frontend (the application used to see the logs) are started. To use it, go to

To create some random log messages, you could run:

$ ./dev-scripts/

(and press Ctrl+C twice to stop it).

The project could be imported from within Eclipse PyDev.

Current iteration goals

Not implemented right now / Ideas / TODOs

  • Failover on client (if one server doesn't respond, try another)

  • Test and compare performance and disk space: StorageService vs StorageService2

  • Document installation procedure

  • Easy deploy using Gunicorn

  • Add tests with Selenium / WebDriver Plus

  • Move Daedalus client to separate project

  • Add check of memcache on status page

  • Accept messages even when lack some field(s)

  • Accept messages sent with POST even if the message is not JSON encoded

  • Filter by date

  • TTL of messages / automatic disposal of old messages

  • Live update of search results

    • a. Update the search results while new messages arrives
  • Tagging of log messages

  • Search by message text

  • Autentication to save messages (backend) and/or to see the messages (frontend)

General architecture

  • Client + server app.

  • Client:

    • Thin layer over a http client to send the messages
  • Server:

    • Backend: Django app that receives the messages.
    • Frontend: Django app for viewing the messages.
  • Messages sent over HTTP

  • Messages encoded using JSON


  • A single keyspace holds all the column families.

  • Messages are stored using 4 column families:

    • CF: Logs - Cols[]: { uuid1/timestamp: JSON encodded message }
    • CF: Logs_by_app - Cols[]: { uuid1/timestamp: JSON encodded message }
    • CF: Logs_by_host - Cols[]: { uuid1/timestamp: JSON encodded message }
    • CF: Logs_by_severity - Cols[]: { uuid1/timestamp: JSON encodded message }
  • Alternative format (implemented by StorageService2):

    • CF: Logs - Cols[]: { uuid1/timestamp: JSON encodded message }
    • CF: Logs_by_app - Cols[]: { uuid1/timestamp: '' }
    • CF: Logs_by_host - Cols[]: { uuid1/timestamp: '' }
    • CF: Logs_by_severity - Cols[]: { uuid1/timestamp: '' }
  • No SuperColumn nor secondary indexes by now.


  • Log message: structure containing:
    • message
    • detail
    • application
    • host
    • severity
    • timestamp (Cassandra)



  • Many enhacements on fabric scripts and new tasks: daedalus_syncdb(), gunicorn_launch()
  • Created dev-scripts/ to document and automatize installation on Ubuntu
  • Updated scripts on dev-scripts/ to automatically use virtualenv if exists


#    daedalus - Centralized log server
#    Copyright (C) 2012 - Horacio Guillermo de Oro <>
#    This file is part of daedalus.
#    daedalus is free software; you can redistribute it and/or modify
#    it under the terms of the GNU General Public License as published by
#    the Free Software Foundation version 2.
#    daedalus is distributed in the hope that it will be useful,
#    but WITHOUT ANY WARRANTY; without even the implied warranty of
#    GNU General Public License version 2 for more details.
#    You should have received a copy of the GNU General Public License
#    along with daedalus; see the file LICENSE.txt.
Something went wrong with that request. Please try again.