Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

单次爬取是否是因为数量的限制呢，目前都是爬到1995条就停止了 #38

Closed
zsr0423 opened this issue Jan 6, 2019 · 3 comments
Closed

单次爬取是否是因为数量的限制呢，目前都是爬到1995条就停止了 #38

zsr0423 opened this issue Jan 6, 2019 · 3 comments

Comments

@zsr0423
Copy link

zsr0423 commented Jan 6, 2019

No description provided.

@zsr0423
Copy link
Author

zsr0423 commented Jan 6, 2019

希望解答

@zsr0423 zsr0423 changed the title 爬取 单次爬取是否是因为数量的限制呢，目前都是爬到1995条就停止了 Jan 6, 2019
@dataabc
Copy link
Owner

dataabc commented Jan 7, 2019 •
edited

没有爬取数量的限制，请问有错误提示吗?
可能是因为爬取速度太快了，可以每爬取一定页数sleep几秒，以此来解决问题

@slimwang
Copy link

一般服务器都会对单位时间内的访问次数做一定的限制
数据量大的话, 建议在不同时间段进行爬取

@dataabc dataabc closed this as completed Mar 27, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@slimwang @dataabc @zsr0423