通过scrapy进行全国工商数据的采集，保存到elasticsearch中去,(一天一台服务器大概300-500万数据）注:需要配合代理ip使用,我这里的使用的代理是阿布云,方法已经写在middlewares中去 es版本5.1.0 scrapy版本1.5.1 python版本3.6 运行方法通过main中来控制
-
Notifications
You must be signed in to change notification settings - Fork 0
kenneth663/scrapy-case
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
About
No description, website, or topics provided.
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published