simple python gevent web spider
Latest commit f3a869c Jun 27, 2011 @jmoiron change the SimpleHandler semantics to issue callbacks with only the j…
…ob, change the crawler to tack on the current crawler to the job during pre processing (so that job callbacks can access the crawler's job queue and add new ones independently of the handler on that job), change simple.startjobs to be able to create a simple handler from a callback so you can pass a BaseHandler or a callable as the handler kwarg



Aranha (pronounced aranya) is a simple web spider written in python using gevent for asynchronicity. Aranha means "spider" in portuguese.

Aranha's goal is to be suitable for projects that need light url fetching or a simple spidering of a few classes of webpages. If spidering is a major part of your project, you probably want to either write your own spider or use scrapy as a base, as it's much more sophisticated.