Skip to content

Navigation Menu

scrapy

Explore
For
- Enterprise
- Teams
- Startups
- Education
By Solution
Resources
- GitHub Sponsors
  Fund open source developers
- The ReadME Project
  GitHub community articles
Repositories
- Enterprise platform
  AI-powered developer platform
Available add-ons
Pricing

Search code, repositories, users, issues, pull requests...

Search

Clear

Search syntax tips

Provide feedback

We read every piece of feedback, and take your input very seriously.

Include my email address so I can be contacted

Saved searches

Use saved searches to filter your results more quickly

Name

Query

To see all available qualifiers, see our documentation.

You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session.

Dismiss alert

Scrapy project

An open source and collaborative framework for extracting the data you need from websites. In a fast, simple, yet extensible way.

462 followers
https://scrapy.org

Overview
Repositories
Projects
Packages
People

More

Overview
Repositories
Projects
Packages
People

Pinned

scrapy scrapy Public

Scrapy, a fast high-level web crawling & scraping framework for Python.

Python 51.4k 10.4k
scrapy.org scrapy.org Public

The scrapy.org website

HTML 60 138

Repositories

Type

Select type

All Public Sources Forks Archived Mirrors Templates

Language

Select language

All C++ DIGITAL Command Language HTML Python Shell

Sort

Select order

Last updated Name Stars

Showing 10 of 26 repositories

scrapy Public
Scrapy, a fast high-level web crawling & scraping framework for Python.

Python 51,377 BSD-3-Clause 10,379 436 (21 issues need help) 220 Updated Jun 10, 2024
parsel Public
Parsel lets you extract data from XML/HTML documents using XPath or CSS selectors

Python 1,094 BSD-3-Clause 135 29 (1 issue needs help) 11 Updated Jun 9, 2024
w3lib Public
Python library of web-related functions

Python 384 BSD-3-Clause 103 12 (1 issue needs help) 5 Updated Jun 5, 2024
itemloaders Public
Library to populate items using XPath and CSS with a convenient API

Python 43 BSD-3-Clause 15 17 4 Updated Jun 4, 2024
itemadapter Public
Common interface for data container classes

Python 60 BSD-3-Clause 10 6 3 Updated Jun 3, 2024
scrapyd Public
A service daemon to run Scrapy spiders

Python 2,867 BSD-3-Clause 567 21 5 Updated May 31, 2024
protego Public
A pure-Python robots.txt parser with support for modern conventions.

DIGITAL Command Language 52 BSD-3-Clause 26 6 (1 issue needs help) 1 Updated May 28, 2024
scrapy.org Public
The scrapy.org website

HTML 60 138 1 1 Updated May 14, 2024
queuelib Public
Collection of persistent (disk-based) and non-persistent (memory-based) queues for Python

Python 262 BSD-3-Clause 53 3 2 Updated May 4, 2024
loginform Public
Fill HTML login forms automatically

Python 267 78 11 3 Updated Apr 24, 2024

View all repositories

People

Top languages

Python C++ HTML Shell DIGITAL Command Language

Most used topics

hacktoberfest python chromium gurl scraping

Footer

© 2024 GitHub, Inc.

Footer navigation

Terms
Privacy
Security
Status
Docs
Contact

You can’t perform that action at this time.