Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
If you're new to Scrapy, start by reading Scrapy at a glance.
Google Summer of Code
Scrappy help docs and resources
- Articles & blog posts
Projects, tools and libraries using Scrapy
- Django Dynamic Scraper - a web application (written in django) for runnning and controlling Scrapy spiders
- Slybot - A supervised learning crawler based on Scrapely
- scrapy-sentry - Logs Scrapy exceptions into Sentry
- ScrapyGraphite - Output scrapy statistics to carbon/graphite
- scrapy-mongo - A pipeline to store scrapy items in a MongoDB database
- scrapy-boilerplate - small set of utilities to simplify writing low-complexity spiders
- scrapy-inline-requests - provides a decorator to write spider callbacks which performs multiple requests without the need to write multiple callbacks for each request
- scrapy-redis - providesRedis-backed components for Scrapy
- scrapyz - Create simple spiders easily.
- Scrapy-related libraries on PyPI
- Scrapy_cn - provided a demo to solve encoding problems(utf-8).
- elite-proxies-scrapy-middleware - get new proxies from your EliteProxies account
- scrapydo - Crochet-based blocking API for Scrapy.
Companies using Scrapy
- see Release notes in the official documentation
Scrapy Enhancement Proposals
- SEPs are available in scrapy/sep.
Clone this wiki locally
Press h to open a hovercard with more details.