scrapy

Scrapy, a fast high-level web crawling & scraping framework for Python.

Updated Aug 30, 2016

Python 98 19

parsel

Parsel lets you extract data from XML/HTML documents using XPath or CSS selectors

Updated Aug 29, 2016

Python 185 47

w3lib

Python library of web-related functions

Updated Aug 25, 2016

scrapyd

A service daemon to run Scrapy spiders

Updated Aug 24, 2016

Python 91 33

scrapyd-client

Command line client for Scrapyd server

Updated Aug 24, 2016

Python 147 45

loginform

Fill HTML login forms automatically

Updated Aug 16, 2016

scrapy.org

Source code for Scrapy website

Updated Aug 15, 2016

dirbot

Scrapy project to scrape public web directories (educational)

Updated Aug 14, 2016

Python 1 1

scrapy-itemloader

Library to populate Scrapy items using XPath and CSS with a convenient API

Updated Jul 26, 2016

Python 130 29

cssselect

CSS Selectors for Python

Updated Jul 16, 2016

scrapely

A pure-python HTML screen-scraping library

Updated Jul 12, 2016

Python 90 26

queuelib

Collection of persistent (disk-based) queues

Updated May 23, 2016

slybot

Updated Apr 27, 2015

Shell 4 1

gsoc2014-integration-tests

GSoC2014 - Scrapy Integration tests project

Updated Mar 18, 2014