Skip to content
@scrapy

Scrapy project

An open source and collaborative framework for extracting the data you need from websites. In a fast, simple, yet extensible way.

Pinned Loading

  1. scrapy Public

    Scrapy, a fast high-level web crawling & scraping framework for Python.

    Python 54.5k 10.7k

  2. scrapy.org Public

    The scrapy.org website

    HTML 62 141

Repositories

Showing 10 of 27 repositories
  • scrapy Public

    Scrapy, a fast high-level web crawling & scraping framework for Python.

    Python 54,465 BSD-3-Clause 10,691 436 (19 issues need help) 181 Updated Mar 8, 2025
  • scrapyd-client Public

    Command line client for Scrapyd server

    Python 773 BSD-3-Clause 145 5 0 Updated Mar 8, 2025
  • protego Public

    A pure-Python robots.txt parser with support for modern conventions.

    DIGITAL Command Language 60 BSD-3-Clause 28 5 (1 issue needs help) 0 Updated Mar 8, 2025
  • parsel Public

    Parsel lets you extract data from XML/HTML documents using XPath or CSS selectors

    Python 1,199 BSD-3-Clause 150 31 (1 issue needs help) 12 Updated Mar 8, 2025
  • itemloaders Public

    Library to populate items using XPath and CSS with a convenient API

    Python 46 BSD-3-Clause 16 17 4 Updated Mar 8, 2025
  • cssselect Public

    CSS Selectors for Python

    Python 293 61 17 4 Updated Mar 8, 2025
  • queuelib Public

    Collection of persistent (disk-based) and non-persistent (memory-based) queues for Python

    Python 275 BSD-3-Clause 55 4 2 Updated Mar 7, 2025
  • scrapy.org Public

    The scrapy.org website

    HTML 62 141 1 1 Updated Mar 7, 2025
  • scrapyd Public

    A service daemon to run Scrapy spiders

    Python 3,009 BSD-3-Clause 573 8 0 Updated Feb 19, 2025
  • w3lib Public

    Python library of web-related functions

    Python 399 BSD-3-Clause 108 11 (1 issue needs help) 4 Updated Feb 16, 2025
View all repositories

Top languages

Loading…

Most used topics

Loading…