New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

是否可以获取ajax加载后的网页 #20

Closed

the404 opened this issue Sep 10, 2017 · 1 comment

the404 commented Sep 10, 2017

网页是在ajax结束之后，才有了内容数据，而我又需要在ajax渲染完成之后爬取，请问是否支持，如果支持的话具体应该怎么配置呢？

Collaborator

zlzforever commented Sep 11, 2017

异步加载的数据有2总方法

第一优先您应该先去分析一下网站的异步请求API, 直接构造请求采集数据(框架支持JsonPath)
第二如果API有加密等导致您无法模拟出正确的请求, 可以把Downloader设置为WebDriverDownloader

zlzforever closed this as completed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment