Skip to content
This repository

Scrapy, a fast high-level screen scraping and web crawling framework for Python.

Merge pull request #692 from kmike/version_info_fix

fix scrapy.version_info when SCRAPY_VERSION_FROM_GIT is set
latest commit e482c6ec55
Mikhail Korobov kmike authored April 17, 2014
Octocat-spinner-32 artwork added artwork files properly now March 20, 2012
Octocat-spinner-32 bin remove scrapyd, it was migrated to its own repository February 06, 2013
Octocat-spinner-32 debian Added "six>=1.5.2" to requirements January 15, 2014
Octocat-spinner-32 docs Fix broken links in documentation April 09, 2014
Octocat-spinner-32 extras remove references to deprecated scrapy-developers list February 16, 2014
Octocat-spinner-32 scrapy fix scrapy.version_info when SCRAPY_VERSION_FROM_GIT is set April 16, 2014
Octocat-spinner-32 sep sep 14 for #629 March 07, 2014
Octocat-spinner-32 .coveragerc Added rules to Makefile.buildbot for generating coverage reports December 15, 2010
Octocat-spinner-32 .gitignore Added request_fingerprint method to dupefilter classes so they could … January 15, 2014
Octocat-spinner-32 building Pillow with pypy requries dev headers November 07, 2013
Octocat-spinner-32 .travis.yml New tox env: docs April 09, 2014
Octocat-spinner-32 AUTHORS added Nicolas Ramirez to AUTHORS March 14, 2013
Octocat-spinner-32 renamed CONTRIBUTING to so that links are rendered as… September 19, 2012
Octocat-spinner-32 INSTALL fix link to online installation instructions October 02, 2012
Octocat-spinner-32 LICENSE mv scrapy/trunk to root as part of svn2hg migration May 06, 2009
Octocat-spinner-32 get scrapy version from package data February 06, 2013
Octocat-spinner-32 Makefile.buildbot Fix permission and set umask before generating sdist tarball September 03, 2013
Octocat-spinner-32 NEWS added NEWS file pointing to docs/news.rst April 28, 2012
Octocat-spinner-32 README.rst Drop Python 2.6 support October 29, 2013
Octocat-spinner-32 pytest.ini Support doctest and test discover in pytest April 04, 2014
Octocat-spinner-32 requirements.txt test_command_deploy, test_contrib_linkextractors January 11, 2014
Octocat-spinner-32 setup.cfg remove no longer existent examples from doc_files used in bdist_rpm. … October 08, 2013
Octocat-spinner-32 Added "six>=1.5.2" to requirements January 15, 2014
Octocat-spinner-32 tests-requirements.txt Run testsuite with py.test April 03, 2014
Octocat-spinner-32 tox.ini New tox env: docs April 09, 2014



Scrapy is a fast high-level screen scraping and web crawling framework, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing.

For more information including a list of features check the Scrapy homepage at:


  • Python 2.7
  • Works on Linux, Windows, Mac OSX, BSD


The quick way:

pip install scrapy

For more details see the install section in the documentation:


You can download the latest stable and development releases from:


Documentation is available online at and in the docs directory.

Community (blog, twitter, mail list, IRC)




Companies using Scrapy


Commercial Support


Something went wrong with that request. Please try again.