Skip to content
🦎 A DHT crawler with a minimalistic frontend, written in Python
Python Shell HTML Dockerfile CSS
Branch: master
Clone or download
Latest commit 91f7d39 Dec 31, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
docker Fix Dec 31, 2019
docs
resources Reescale stilio logo images Oct 16, 2019
stilio Travis fixes Oct 19, 2019
tests Initial commit Oct 9, 2019
.gitignore Change docker config Dec 31, 2019
.travis.yml Travis fixes Oct 19, 2019
LICENSE Update license year Oct 9, 2019
Pipfile Simplify local development Oct 18, 2019
Pipfile.lock Simplify local development Oct 18, 2019
README.rst Text changes Dec 31, 2019
dev.yml Change docker config Dec 31, 2019
docker-compose.yml Change docker config Dec 31, 2019
env-example Switch to postgres and peewee Oct 13, 2019
mypy.ini Travis fixes Oct 19, 2019

README.rst

stilio

https://travis-ci.org/seik/stilio.svg?branch=master https://readthedocs.org/projects/stilio/badge/?version=latest https://img.shields.io/github/license/seik/stilio

Goals

  • Completely written in python.
  • Fast. But *never* trade code readability for speed.
  • Easy set up.

Quickstart

A docker-compose file is included to allow a fast and easy set up.

docker-compose up

Note that a it's recommended to run the frontend and database separated from the crawler since the last one generates a lot of traffic and can make a poor experience for the users. You can read a guide about how to do this here.

FAQ

The crawler speed is not so good.

Try to run this commands if you are on linux:

iptables -I OUTPUT -t raw -p udp --sport PORT_NUMBER -j NOTRACK
iptables -I PREROUTING -t raw -p udp --dport PORT_NUMBER -j NOTRACK
You can’t perform that action at this time.