bianjiang/tweetf0rm

handle dead crawler gracefully: cleanup residue resources; and schedu… …

…le it to restart after 30 mins

clean up leftover, if a crawler gets restarted

close redis connection when its done; otherwise, it causes error

Merge branch 'master' of bitbucket.org:bianjiang/tweetf0rm

add in function to get the full user object from a list of user ids; … …

…and add in GET_USERS_FROM_IDS command; see tweetf0rm/client.py;

change python2.7 to python in boostrap.sh and client.sh

update readme, and sample config file

killall.sh script

use app_key as id, and add-in support for restart the entire crawler

save empty file as well so we can track progress

adding auto-gzip to archived folder support

remove proxies.json and add in proxies_sample.json

remove passwrod from config sample

remove config data

update dataset

timestamp on logging

more debug code to diagnosis

show full stack when error occurs, so we can catch what's wrong..

debug message to find out bad status error

remove noses

complete bootstrap.sh, and client.sh

proxies.json

add hash value for cmd

cmd_handlers specific for control adding new cmds based on the depth … …

…of the network you want to crawl

get rid of duplicated handlers

duplicates

user_id is not necessary always be numbers in the code

fix depth number problem

fix depth number problem

handle non-numbers

handle non-numbers

client.py and README.md that explains the usage of client.py

pprint status

deal with Auth Error during start

list_alive()