twitter search result store on CouchDB
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.

What is TStore

TStore is a twitter search result backup application build on Apache CouchDB.


  • CouchDB 0.10.0 or higher
  • Python 2.5 or higher


Install Prerequistic Libraries

Install following libraries using easy_install

  • couchapp

Note that we use couchapp, not official version but modified version.

Install CouchApp application on your CouchDB

Download the source repositories

$ git clone git://
$ cd tstore
$ git clone git:// search/vendor/crayon
$ git clone git:// vendor/yssk22-couchapp
$ chmod 755 ../vendor/yssk22-couchapp/bin/couchapp

Deploy TStore on your CouchDB

$ cd search
$ cp search/dot_couchapprc couchapp/.couchapprc
$ vi search/.couchapprc
$ export PYTHONPATH=../vendor/yssk22-couchapp
$ ../vendor/yssk22-couchapp/bin/couchapp push 

Configure twitter search crawler

Configure the crawler task to store search results on CouchDB.

Set task script perission

$ chown couchdb.couchdb search/_attachments/jobs/crawler/
$ chown 700 search/_attachments/jobs/crawler/

Prepare crawler configuration file

You need to preprare json-formatted configuration file for crawler jobs as follows:

   "couchdb": {              
     "host" : "localhost",
     "port" : 5984,
     "db"   : "tstore_default",
     "user" : "admin",
     "password" : "time2relax"
   "threads" : {              
     "wait_interval"  : 300,  
     "max_crawlers"   : 5

Save this to HOME/tstore-search.json (or any directory that can be acceessed by your CouchDB OS user) and then

$ chmod 600 tstore-search.json
$ chown couchdb.couchdb tstore-search.json

Note that twitter search API has

configure external API

Configure your local.ini file for CouchDB.

ts-search-crawler = /opt/local/bin/python TSTORE_ROOT/search/_attachments/jobs/crawler/ -f HOME/ts-search-config.json 2> /tmp/ts-search-config.log

_ts-search-crawler = {couch_httpd_external, handle_external_req, <<"ts-search-crawler">>}

See demo