Skip to content
Scrapy, a fast high-level web crawling & scraping framework for Python.
Python Other
Latest commit 4158839 @redapple redapple Merge pull request #1765 from scrapy/add-deprecation-for-pydispatch
[MRG+1] Add fallback and deprecation warning for pydispatch (fixes #1762)
Failed to load latest commit information.
artwork added artwork files properly now
debian Merge pull request #934 from Dineshs91/zsh-support
docs add deprecation for pydispatch (thanks for the help @redapple)
extras Merge pull request #934 from Dineshs91/zsh-support
scrapy Merge pull request #1765 from scrapy/add-deprecation-for-pydispatch
sep Spelling fixes
tests Merge pull request #1765 from scrapy/add-deprecation-for-pydispatch
.bumpversion.cfg Bump version: 1.2.0dev1 → 1.2.0dev2
.coveragerc Add coverage report trough
.gitignore add coverage files to gitignore
.travis.yml Enable travis builds on tag patterns
AUTHORS added Nicolas Ramirez to AUTHORS Add Code of Conduct Version 1.3.0 from Put a blurb about support channels in CONTRIBUTING
INSTALL fix link to online installation instructions
LICENSE mv scrapy/trunk to root as part of svn2hg migration ENH: include tests/ to source distribution in
Makefile.buildbot Generated version as pep440 and dpkg compatible
NEWS added NEWS file pointing to docs/news.rst
README.rst Add link to CoC mardown file on Github Simplify if statement
pytest.ini Don't collect tests by their class name
requirements-py3.txt raise minimal twisted version for py3
requirements.txt upgrade parsel and add shim for deprecated selectorlist methods
setup.cfg remove no longer existent examples from doc_files used in bdist_rpm. … upgrade parsel and use its function to instantiate root for finding form
tox.ini Add Python 3.5 tox env and Python 3.4 tests in Travis CI



PyPI Version PyPI Monthly downloads Build Status Wheel Status Python 3 Porting Status Coverage report


Scrapy is a fast high-level web crawling and web scraping framework, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing.

For more information including a list of features check the Scrapy homepage at:


  • Python 2.7
  • Works on Linux, Windows, Mac OSX, BSD


The quick way:

pip install scrapy

For more details see the install section in the documentation:


You can download the latest stable and development releases from:


Documentation is available online at and in the docs directory.

Community (blog, twitter, mail list, IRC)



Please note that this project is released with a Contributor Code of Conduct (see

By participating in this project you agree to abide by its terms. Please report unacceptable behavior to


Companies using Scrapy


Commercial Support


Something went wrong with that request. Please try again.