Skip to content
Flask and Scrapy example site.
Python HTML JavaScript CSS
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
scrapy
static
templates
.gitignore
LICENSE
README.md
add_categories.py
requirements.txt
server.py

README.md

taobaobao

该项目使用 Scrapy 抓取 taobao 商品信息,并存入 MongoDB 数据库。

Web 端使用 Flask + Bootstrap 构建聚合搜索商品信息的简单应用。

deployment

安装 MongoDB 及相关 Python 依赖。

$ sudo pip install -r requirements.txt
$ sudo dnf install mongodb-server mongodb mongo-tools
$ sudo systemctl start mongod.service

运行 Scrapy 爬取商品信息。

$ cd scrapy
$ scrapy crawl taobao

获取信息后,运行 Web Server 进行测试。

$ python server.py
 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
 * Restarting with stat
 * Debugger is active!
$ firefox http://127.0.0.1:5000/
You can’t perform that action at this time.