Proxymesh downloader middleware for Scrapy
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
scproxymesh Add proxy rotation and auth support Jan 5, 2017
LICENSE Initial commit Nov 24, 2015
README.md Fix typo Jan 23, 2017
setup.py Add proxy rotation and auth support Jan 5, 2017

README.md

scrapy-proxymesh

Proxymesh downloader middleware for Scrapy.

Installation

From PyPI
pip install scrapy-proxymesh
From GitHub
pip install -e git+https://github.com/mizhgun/scrapy-proxymesh@master#egg=scproxymesh

Usage

settings.py:

DOWNLOADER_MIDDLEWARES = {
    'scproxymesh.SimpleProxymeshMiddleware': 100,
}

PROXYMESH_URL = 'http://us-il.proxymesh.com:31280'

# Proxymesh request timeout
PROXYMESH_TIMEOUT = 60