Scrapy parse doesn't support async callbacks with yield

[GKukulski]

When running scrapy parse http://localhost:8080 -c parse --spider test is rising error: TypeError: 'async_generator' object is not iterable.

Simple Code of Spider:

import scrapy

class TestSpider(scrapy.Spider):
    name = 'test'
    allowed_domains = ['localhost']
    start_urls = ['http://localhost:8080']

    async def parse(self, response):
        for x in range(1, 5):
            yield ({"test": x})

It is happened when there is yield inside async function.

scrapy crawl test - works fine

[Gallaecio]

I can confirm this issue, and I don’t think it is a regression, because even with the commit that is meant to fix it, and includes a test for it, I can reproduce the issue (commit 0435751, and passing an extra command line to get that commit working: -s REQUEST_FINGERPRINTER_IMPLEMENTATION=PREVIOUS_VERSION).

[wRAR]

The PR (#5577) indeed only tests async def callbacks without a yield (explicitly, as that test callback returns a list).