@scrapy-plugins

Scrapy Plugins

Plugins for the Scrapy framework

  • Crawlera middleware for Scrapy

    Python 32 15 Updated Jan 21, 2017
  • Scrapy schema validation pipeline and Item builder using JSON Schema

    Python 1 1 Updated Jan 20, 2017
  • Scrapy spider middleware to use Scrapinghub's Hub Crawl Frontier as a backend for URLs

    Python 1 2 Updated Jan 18, 2017
  • Scrapy extension to control spiders using JSON-RPC

    Python 94 33 Updated Jan 18, 2017
  • Scrapy+Splash for JavaScript integration

    Python 680 163 Updated Dec 20, 2016
  • A scrapy extension to store requests and responses information in storage service

    Python 4 Updated Dec 16, 2016
  • Scrapy spider middleware to ignore requests to pages containing items seen in previous crawls

    Python 32 7 Updated Dec 7, 2016
  • A Scrapy pipeline to categorize items using MonkeyLearn

    Python 26 8 Updated Nov 1, 2016
  • Python 9 1 Updated Oct 12, 2016
  • A scrapy extension to sync `.scrapy` folder to an S3 bucket

    Python 1 1 Updated Aug 4, 2016
  • Scrapy middleware to add extra fields to items, like timestamp, response fields, spider attributes etc.

    Python 9 1 Updated Jul 20, 2016
  • Scrapy spider middleware to split an item into multiple items using a multi-valued key

    Python 2 Updated Jul 18, 2016
  • Scrapy spider middleware to clean up query parameters in request URLs

    Python 4 Updated Jun 30, 2016
  • Scrapy extension to write scraped items using Django models

    Python 109 14 Updated May 4, 2016
  • Python 3 1 Updated Jan 26, 2016
  • Scrapy pipeline for writing items to BigML datasets

    Python 1 Updated Nov 17, 2015
  • Scrapy support for working with streamcorpus Stream Items.

    Python 7 Updated Jan 8, 2015