Skip to content

v0.6.2a3

Pre-release
Pre-release

Choose a tag to compare

@fawadss1 fawadss1 released this 19 May 11:55
· 116 commits to master since this release

Fixed

  • All Scrapy versions — unified async dispatch in BaseEngine.fetch
    Scrapy routes async downloader middlewares through one of two runners depending on
    the version, and each runner requires a different awaitable type:

    Scrapy path Runner Needs
    ensure_awaitable (newer Scrapy, local) asyncio Task asyncio.Future (run_in_executor)
    deferred_from_coro (Zyte scrapy:2.11–2.12) Twisted _inlineCallbacks Twisted Deferred (deferToThread)

    BaseEngine.fetch now detects the active runner via asyncio.get_running_loop():
    if a running loop is found (asyncio Task context) it uses loop.run_in_executor;
    if RuntimeError is raised (no asyncio Task, Twisted _inlineCallbacks context)
    it falls back to deferToThread. This replaces the single-strategy implementations
    from a2 that each fixed one path but broke the other.