SIRC - "Search IRC"
SIRC is a search engine for IRC logs. It has four big parts:
- The front end, which lets users do searches and shows the results (and lets users browse old logs).
- The back end, which indexes IRC logs and stores them in the log repository.
- The search index.
- The log repository.
The front end
The front end is written in Python and runs on Google's App Engine. You can see it in action (try it out!) at http://heavymetalab.appspot.com/.
The front end queries the search index for searches, displays search results, and renders logs for display.
The home page:
The results of a search:
Looking at the log for one channel for one day:
Browsing the entire history of a channel, with color-coded activity indicators:
The back end
The back end collects IRC logs, adds them to the index, and stores them in the log repository.
The search index
SIRC indexes each line of every IRC log using Solr.
The log repository
I store the logs in Amazon S3. The front end gets them from S3, marks them up with HTML, and shows them to the user.