Skip to content
dhtcrawler is a DHT crawler written in erlang. It can join a DHT network and crawl many P2P torrents.
Erlang Shell
Latest commit b8a486d Oct 26, 2013 @kevinlynx Merge pull request #1 from zhkzyth/master
add missing instruction to readme

README.md

dhtcrawler

dhtcrawler is a DHT crawler written in erlang. It can join a DHT network and crawl many P2P torrents. The program save all torrent info into database and provide an http interface to search a torrent by a keyword.

screenshot

Usage

  • Download mongodb and start it with text search, i.e:

    mongod --dbpath db --setParameter textSearchEnabled=true
    
  • Download dhtcrawler source code

  • Use rebar to download and install all dependent libraries

    rebar get-deps
    
  • compile

    rebar compile
    
  • starting the erlang shell with the ebin path added

    erl -pa ebin
    
  • start dhtcrawler

    crawler_app:start()
    
  • start the http front-end

    crawler_http:start().
    
  • Open a web browser and point to localhost:8000/index.html

Config

see priv/dhtcrawler.config.

NOTE, when you change node_count value in dhtcrawler.config, you'd better delete all files saved in dhtstate directory.

Something went wrong with that request. Please try again.