Spiders_Collection

Save all spiders I've wrote

asyncio不支持request库，可使用aiohttp.

baiduTranslateSignJson.js 这个文件是生成sign的json函数文件.

chromedriver在浏览器版本88.0之后设置enable-automation反检测的手段失效，首页中有新解决方案.

-手动处理：通过抓包工具获取cookie值并封装到request请求的headers中
-自动处理：创建session对象：requests.Session()

-使用session对象进行模拟登录post请求的发送(cookie会存储在session中)

-session对象对个人主页对应的get请求进行发送（携带cookie)

Name		Name	Last commit message	Last commit date
Latest commit History 111 Commits
07.ChinaZ_Jianli		07.ChinaZ_Jianli
News163_Spider		News163_Spider
No_11_sunPro		No_11_sunPro
No_12_fbsPro		No_12_fbsPro
bossPro		bossPro
imgsPro		imgsPro
qiubaiPro		qiubaiPro
qiutuLibs		qiutuLibs
test_scrapy		test_scrapy
多线程爬虫学习		多线程爬虫学习
.gitignore		.gitignore
01.sogouSpider.py		01.sogouSpider.py
02.kfcSpider.py		02.kfcSpider.py
03._baiduTranslateSignJson.js		03._baiduTranslateSignJson.js
03.baiduTranslateSpider.py		03.baiduTranslateSpider.py
03.result_json.js		03.result_json.js
04.NMPA_Spider.py		04.NMPA_Spider.py
04.NMPA_Spider_Result.js		04.NMPA_Spider_Result.js
05.qiushibaike_Image_Spider.py		05.qiushibaike_Image_Spider.py
06.58ershoufang_Spider.py		06.58ershoufang_Spider.py
06._58ershou.txt		06._58ershou.txt
07.ChinaZ_Spider.py		07.ChinaZ_Spider.py
08.CAPTCHA.jpg		08.CAPTCHA.jpg
08.gushiwen_Anti-CAPTCHA.py		08.gushiwen_Anti-CAPTCHA.py
09.proxy.py		09.proxy.py
10.12306_Simulated_login.py		10.12306_Simulated_login.py
10.CAPTCHA.png		10.CAPTCHA.png
10.screenshot.png		10.screenshot.png
LICENSE		LICENSE
README.md		README.md
js.html		js.html
test_anjuke_selenium.py		test_anjuke_selenium.py
test_bs4.py		test_bs4.py
test_re.py		test_re.py
test_selenium_NMPA.py		test_selenium_NMPA.py
test_selenium_boss.py		test_selenium_boss.py
test_selenium_headless_browser.py		test_selenium_headless_browser.py
test_selenium_taobao.py		test_selenium_taobao.py
test_xpath_example.py		test_xpath_example.py