We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
i want to use scrapy-splash with proxy,my code like this: def start_requests(self): for url in self.start_urls: yield SplashRequest(url, self.parse,args={'wait': 10,'proxy': 'http://127.0.0.1:1080'}) but i get some messsage in console like this: 502 Bad Gateway and splash show some message : 2018-07-10 16:00:16.670873 [events] {"qsize": 0, "load": [0.02, 0.04, 0.0], "rendertime": 0.02126336097717285, "error": {"type": "RenderError", "info": {"text": "Proxy connection refused", "code": 99, "url": "https://www.baidu.com", "type": "Network"}, "description": "Error rendering page", "error": 502}, "path": "/render.html", "_id": 139738842022520, "fds": 20, "args": {"proxy": "http://127.0.0.1:1081", "wait": 10, "headers": {"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8", "User-Agent": "Scrapy/1.5.0 (+https://scrapy.org)", "Accept-Language": "en"}, "uid": 139738842022520, "url": "https://www.baidu.com"}, "maxrss": 149472, "user-agent": "Scrapy/1.5.0 (+https://scrapy.org)", "method": "POST", "client_ip": "172.17.0.1", "timestamp": 1531238416, "status_code": 502, "active": 0} 2018-07-10 16:00:16.672596 [-] "172.17.0.1" - - [10/Jul/2018:16:00:15 +0000] "POST /render.html HTTP/1.1" 502 269 "-" "Scrapy/1.5.0 (+https://scrapy.org)" who can help me ,please
The text was updated successfully, but these errors were encountered:
i met such kind of issue, and found it was docker splash issue.
Sorry, something went wrong.
I think it might because the bad ip proxy, try to change to another one.
@AceXia Did you ever find a solution?
No branches or pull requests
i want to use scrapy-splash with proxy,my code like this:
def start_requests(self):
for url in self.start_urls:
yield SplashRequest(url, self.parse,args={'wait': 10,'proxy': 'http://127.0.0.1:1080'})
but i get some messsage in console like this: 502 Bad Gateway
and splash show some message :
2018-07-10 16:00:16.670873 [events] {"qsize": 0, "load": [0.02, 0.04, 0.0], "rendertime": 0.02126336097717285, "error": {"type": "RenderError", "info": {"text": "Proxy connection refused", "code": 99, "url": "https://www.baidu.com", "type": "Network"}, "description": "Error rendering page", "error": 502}, "path": "/render.html", "_id": 139738842022520, "fds": 20, "args": {"proxy": "http://127.0.0.1:1081", "wait": 10, "headers": {"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8", "User-Agent": "Scrapy/1.5.0 (+https://scrapy.org)", "Accept-Language": "en"}, "uid": 139738842022520, "url": "https://www.baidu.com"}, "maxrss": 149472, "user-agent": "Scrapy/1.5.0 (+https://scrapy.org)", "method": "POST", "client_ip": "172.17.0.1", "timestamp": 1531238416, "status_code": 502, "active": 0}
2018-07-10 16:00:16.672596 [-] "172.17.0.1" - - [10/Jul/2018:16:00:15 +0000] "POST /render.html HTTP/1.1" 502 269 "-" "Scrapy/1.5.0 (+https://scrapy.org)"
who can help me ,please
The text was updated successfully, but these errors were encountered: