Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can I get remote server's ip address via response? #3903

Closed
imfht opened this issue Jul 25, 2019 · 3 comments · Fixed by #3940
Closed

Can I get remote server's ip address via response? #3903

imfht opened this issue Jul 25, 2019 · 3 comments · Fixed by #3940

Comments

@imfht
Copy link

@imfht imfht commented Jul 25, 2019

Can I get remote server's ip address via response?
For some reason. I'll need get remote site's ip address when parsing response. I looked the document but found nothing.
Any one know that?
Thanks!

@anubhavp28
Copy link
Contributor

@anubhavp28 anubhavp28 commented Jul 30, 2019

You could use the url property of the Response object, pass it to urllib.parse.urlparse fuction to obtain the domain name. Then you can pass the domain to socket.gethostbyname to get the IP address. I haven't tested it, but it should work.

I think Stack Overflow is a better place to ask questions of this kind than GitHub. GitHub's issue tracker is primarily for bug reports and feature requests.

@Gallaecio
Copy link
Member

@Gallaecio Gallaecio commented Aug 1, 2019

I don’t think this is possible at the moment (in a straightforward way, at least), but it sounds like a valid feature request.

@imfht
Copy link
Author

@imfht imfht commented Aug 5, 2019

Thanks @anubhavp28 .
I don't think it is the right way to get remote ip in scrapy
get_host_by_name cause another new dns resolve. Actually we've got remote ip in scrapy.
class CrawlerProcess has an built-in dnscache object but we can not get it in pipeline.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants