利用cookie登录豆瓣，然后抓取豆瓣上所有小组，总共8个大类67个小类，9w左右个小组。
cookie_str2dict.py 用于将浏览器上的cookie字符串转化为字典键值对
使用scrapy框架，用CrawlSpider 进行爬取
使用mongodb 存储数据
-
Notifications
You must be signed in to change notification settings - Fork 1
利用cookie登录豆瓣，然后抓取豆瓣上所有小组，总共8个大类67个小类，9w左右个小组。
License
OneTraTown/douban_group
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
About
利用cookie登录豆瓣，然后抓取豆瓣上所有小组，总共8个大类67个小类，9w左右个小组。
Resources
License
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published