Ebot, an Opensource Web Crawler built on top of a nosql database (apache couchdb, riak), AMQP database (rabbitmq), webmachine and mochiweb. Ebot is written in Erlang and it is a very scalable, distribuited and highly configurable web cawler. See wiki pages for more details
Erlang Shell
Pull request Compare This branch is 6 commits ahead of matteoredaelli:master.
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
deps
doc
ebin
priv
src
utils
.gitignore
.hgignore
Emakefile
LICENSE.txt
Makefile
README
start-dev.sh
start.sh
sys.config

README

EBOT (http://www.redaelli.org/matteo-blog/projects/ebot/)
USAGE: see the wiki homepage (http://wiki.github.com/matteoredaelli/ebot/) and ebot_test.erl for details
LICENSE: GPL v3+, see file LICENSE
Copyright (c) 2010, 2011 - Matteo Redaelli at libero dot it

Tested with Linux (Ubuntu and Debian) and Erlang R13B03 and R14A

*******************************************************************************
* DOWNLOAD "TAG" releases and not the latest sources that could be unstable...
*******************************************************************************


=== Compliling ===

By now, you need to fill the ./deps by hands. Resulting deps should be like this:

$ ls -l ./deps
/couchbeam			(git://github.com/benoitc/couchbeam.git)
/rabbitmq-erlang-client		(hg clone http://hg.rabbitmq.com/rabbitmq-erlang-client/)
/rabbit_common			symlink to ./rabbitmq-erlang-client/dist/rabbit_common-0.0.0
/webmachine			(git://github.com/basho/webmachine.git)
/riak-erlang-client		(git://github.com/basho/riak-erlang-client.git)
/ibrowse			(git://github.com/cmullaparthi/ibrowse.git)

See https://github.com/matteoredaelli/ebot/wiki/Installation for more info about compiling.


=== Architecture ===

https://github.com/matteoredaelli/ebot/wiki/


=== More info ===

https://github.com/matteoredaelli/ebot/wiki/_pages