Find file History
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
..
Failed to load latest commit information.
movies
README.md
scrapy.cfg

README.md

##豆瓣电影爬虫

###Store

MongoDB
后续可能增加MySQL支持,及用redis作过滤

###dependence

pip install scrapy
pip install pymongo

###Run

cd /movieman/crawler/
scrapy crawl douban

未来将提供更多网站支持