Intelligent proxy pool for Humans™
Clone or download
Latest commit be0e1fe Aug 17, 2018
Failed to load latest commit information.
.github/ISSUE_TEMPLATE Update issue templates [skip ci] May 26, 2018
docs Fix typo Jun 1, 2018
docs_zh Fix doc [skip ci] May 30, 2018
frontend/src Implement Web UI filter for https/anonymous May 29, 2018
scylla Dump 1.1.4 in relation to a new provider #37 Jun 7, 2018
tests Add as provider Jun 7, 2018
.codecov.yml Ignore docs and frontend May 12, 2018
.flake8.cfg Add style check, thanks to #27 May 31, 2018
.gitignore Fix ip checking issue, closes #30 Jun 5, 2018
.readthedocs.yml Remove pdf build for the docs May 30, 2018
.travis.yml Add style check, thanks to #27 May 31, 2018 Create [skip ci] May 26, 2018 Create [skip ci] May 26, 2018
Dockerfile Fix: buyy alpine docker image by using debian May 31, 2018
Dockerfile-alpine Fix: buyy alpine docker image by using debian May 31, 2018
LICENSE Initial commit Apr 10, 2018 Clean up May 25, 2018
Makefile Add test task in Makefile Jun 7, 2018 Create [skip ci] May 26, 2018 Update Aug 2, 2018
README.rst Fix typo Jun 1, 2018
package.json Implement Geometric Distribution, closes #14 May 25, 2018
requirements.txt Fix requirements.txt May 30, 2018
setup.cfg Implement: Web UI in production mode, closes #9 May 25, 2018 Fix issue, dump 1.1.2 May 31, 2018
tsconfig.json Fix tsconfig.json May 25, 2018
yarn.lock Implement Geometric Distribution, closes #14 May 25, 2018

banner_scylla Build Status codecov Documentation Status PyPI version Docker Build Status Donate

An intelligent proxy pool for humanities, only supports Python 3.6. Key features:

  • Automatic proxy ip crawling and validation
  • Easy-to-use JSON API
  • Simple but beautiful web-based user interface (eg. geographical distribution of proxies)
  • Get started with only 1 command minimally
  • Simple HTTP Forward proxy server
  • Scrapy and requests integration with only 1 line of code minimally
  • Headless browser crawling

对于偏好中文的用户,请阅读 中文文档。For those who prefer to use Chinese, please read the Chinese Documentation.

Get started


Install with Docker (highly recommended)

docker run -d -p 8899:8899 -p 8081:8081 -v /var/www/scylla:/var/www/scylla --name scylla wildcat/scylla:latest

Install directly via pip

pip install scylla
scylla --help
scylla # Run the crawler and web server for JSON API

Install from source

git clone
cd scylla

pip install -r requirements.txt

npm install # or yarn install
make build-assets

python -m scylla


This is an example of running a service locally (localhost), using port 8899.

Note: You might have to wait for 1 to 2 minutes in order to get some proxy ips populated in the database for the first time you use Scylla.


Proxy IP List


Optional URL parameters:

Parameters Default value Description
page 1 The page number
limit 20 The number of proxies shown on each page
anonymous any Show anonymous proxies or not. Possible values:true, only anonymous proxies; false, only transparent proxies
https any Show HTTPS proxies or not. Possible values:true, only HTTPS proxies; false, only HTTP proxies
countries None Filter proxies for specific countries. Format example: US, or multi-countries: US,GB

Sample result:

    "proxies": [{
        "id": 599,
        "ip": "",
        "port": 53281,
        "is_valid": true,
        "created_at": 1527590947,
        "updated_at": 1527593751,
        "latency": 23.0,
        "stability": 0.1,
        "is_anonymous": true,
        "is_https": true,
        "attempts": 1,
        "https_attempts": 0,
        "location": "54.0451,-0.8053",
        "organization": "AS57099 Boundless Networks Limited",
        "region": "England",
        "country": "GB",
        "city": "Malton"
    }, {
        "id": 75,
        "ip": "",
        "port": 8080,
        "is_valid": true,
        "created_at": 1527590676,
        "updated_at": 1527593702,
        "latency": 268.0,
        "stability": 0.3,
        "is_anonymous": true,
        "is_https": true,
        "attempts": 1,
        "https_attempts": 0,
        "location": "32.3706,-90.1755",
        "organization": "AS7922 Comcast Cable Communications, LLC",
        "region": "Mississippi",
        "country": "US",
        "city": "Jackson"
    "count": 1025,
    "per_page": 20,
    "page": 1,
    "total_page": 52

System Statistics


Sample result:

    "median": 181.2566407083,
    "valid_count": 1780,
    "total_count": 9528,
    "mean": 174.3290085201

HTTP Forward Proxy Server

By default, Scylla will start a HTTP Forward Proxy Server on port 8081. This server will select one proxy updated recently from the database and it will be used for forward proxy. Whenever an HTTP request comes, the proxy server will select a proxy randomly.

Note: HTTPS requests are not supported at present.

The example for curl using this proxy server is shown below:

curl -x

You could also use this feature with requests:

requests.get('', proxies={'http': ''})

Web UI

Open http://localhost:8899 in your browser to see the Web UI of this project.

Proxy IP List




Globally Geographical Distribution Map




API Documentation

Please read Module Index.


Please see Projects.

Development and Contribution

git clone
cd scylla

pip install -r requirements.txt

npm install # or `yarn install`
make build-assets


If you wish to run tests locally, the commands are shown below:

pip install -r tests/requirements-test.txt
pytest tests/

You are welcomed to add more test cases to this project, increasing the robustness of this project.

Naming of This Project

Scylla is derived from the name of a group of memory chips in the American TV series, Prison Break. This project was named after this American TV series to pay tribute to it.


How to install Python Scylla on CentOS7


If you find this project useful, could you please donate some money to it?

No matter how much the money is, Your donation will inspire the author to develop new features continuously! 🎉 Thank you!

The ways for donation are shown below:



Alipay or WeChat Pay

Alipay And WeChat Donation


Apache License 2.0. For more details, please read the LICENSE file.