Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

spider.request is not awaitable #73

Closed
hiancdtrsnm opened this issue Aug 25, 2019 · 1 comment
Closed

spider.request is not awaitable #73

hiancdtrsnm opened this issue Aug 25, 2019 · 1 comment

Comments

@hiancdtrsnm
Copy link

I just start to use Ruia and I most say that I love it.
But I find my self a little confused with some code in the documentation not working.

The link of the documentation page is https://howie6879.github.io/ruia/en/tutorials/spider.html.

I am trying to replicate:

class MySpider(Spider):
    async def parse(self, response):
        for i in range(10):
            response = await self.request(f'https://some.site/{i}')
            yield self.parse_next(response)

    async def parse_next(self, response):
        print(response.html)

and when give an error when calling self.request and say it's not awaitable

I take a look to the source (installed on my computer) and in did spider.request now returns a Response object.
My question is how to proceed now to get the same result of this line response = await self.request(f'https://some.site/{i}') and be able to manipulate the response

@hiancdtrsnm
Copy link
Author

After experimenting a little I find the solution (call the fetch method of Request object)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant