Skip to content

Utilize Scrapy's native async runners - AsyncCrawlerRunner and/or AsyncCrawlerProcess #638

@vdusek

Description

@vdusek

Following the Scrapy discussion #6639, Scrapy 2.13 introduced new async runners - AsyncCrawlerRunner and AsyncCrawlerProcess (see docs). These classes provide native coroutine-based APIs for running Scrapy projects in asynchronous contexts.

We can adopt these new runners in place of our custom implementations:

Utilizing the built-in Scrapy async runners should simplify our Apify-Scrapy integration, reduce maintenance overhead, and ensure compatibility with future Scrapy releases.

Metadata

Metadata

Assignees

Labels

t-toolingIssues with this label are in the ownership of the tooling team.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions