Skip to content
/ spider Public

🐛 基于静态页面和动态数据的爬虫项目,实现了多线程爬取和一些常见的反扒策略

Notifications You must be signed in to change notification settings

gyoomi/spider

Repository files navigation

项目开发手册

一、项目环境

  • Python3.6+

二、依赖安装

pip install requests -i https://pypi.tuna.tsinghua.edu.cn/simple
pip install scrapy -i https://pypi.tuna.tsinghua.edu.cn/simple
pip install peewee -i https://pypi.tuna.tsinghua.edu.cn/simple
pip install selenium -i https://pypi.tuna.tsinghua.edu.cn/simple
pip install fake-useragent -i https://pypi.tuna.tsinghua.edu.cn/simple

About

🐛 基于静态页面和动态数据的爬虫项目,实现了多线程爬取和一些常见的反扒策略

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published