Narcissus - Realtime visualizations of web server hits
What is this?
narcissus. It is a web application that visualizes web server hits
as they happen in real time.
- IP addresses converted to latitude/longitude, then streamed via orbited and plotted on polymaps.
- Realtime graphs of url-hit distribution with jQuery flot.
- Colorized raw logs streamed to the browser by way of ccze and ansi2html.
- AMQP (Advanced Message Queueing Protocol) from logs to browser.
- Fast. Little to no handshaking overhead.
narcissus was written on April 15th, 2011 at the hackfest hosted by the
FOSSBox at Rochester Institute of Technology.
We knew that mirror.rit.edu was cool, fast, and was
the site with the heaviest traffic on RIT's entire network, but looking at a
mirror frontpage or silently updating your Linux distribution in the background
is not very flashy--not something to phone home about. We wanted to make
something cool. Something that would show the 'big picture' of the Open Source
world. We feel we did that with
The RIT FOSSbox is the launch pad for all things Free & Open Source Software (FOSS) at RIT. It is parented both by the Center for Student Innovation (physically) and the Lab for Technological Literacy's FOSS@RIT Initiative (virtually). They have great staff, mentors, and students who all work on amazing stuff. If you read their history you can learn all sorts of neat things about their work on the One Laptop Per Child project.
narcissus authors got together and got the proof-of-concept (and some
bling-bling) working in a single afternoon but really it was the space, time,
and environment provided by the FOSSbox that made it possible.
This project could not have happened without the hard work of Remy DeCausemaker, the "FOSSBoss" and backbone of the FOSSbox. He and his students have made countless contributions at cost to self that push the limits in the art of software development and code culture on campus, and should be recognized therefore. Together with Luke Macken, RJ Bean, the power of Moksha, and the support of the Grey Beards, RIT has become a real center of gravity for the FOSS Movement.
You can see
narcissus running live at narcissus.rc.rit.edu.
The Research Computing department at the Rochester Institute of Technology runs
narcissus instance. It is pointed at their very active FOSS mirror. (That site is the highest-traffic site at RIT!)
Get the source from github.com.
On your machine that will host narcissus, say,
~/.moksha/ctl.conf with the following content:
narcissus_source_location = /home/threebean/narc/narcissus moksha_source_location = /home/threebean/narc/moksha
Then run the following commands:
$ mkdir narc && cd narc $ git clone git://github.com/ralphbean/narcissus.git $ git clone git://github.com/ralphbean/moksha.git $ pushd moksha && git checkout dev && popd $ pushd narcissus $ ./narc-ctl.py bootstrap $ ./narc-ctl.py rebuild $ ./narc-ctl.py start $ ./narc-ctl.py wtf $ ./narc-ctl.py logs
Finally, on the machine that is being monitored, say,
run the following to setup the narcissus sending script:
$ sudo su - $ yum install inotail python-qpid $ mkdir narc && cd narc $ git clone git://github.com/ralphbean/narcissus.git
And to run it and send stuff to your monitoring host:
$ inotail -f /var/log/lighttpd/access.log | \ ./narcissus/scripts/amqp-log-sender.py --target=monitoring.host.org
inotail is faster than
- Watch out for iptables on ports 9000, 5672, 8080, and 8000.
Try us in
- Ralph Bean <firstname.lastname@example.org>
- Luke Macken <email@example.com>
- Lee Burton <firstname.lastname@example.org>