Skip to content
phpSpider
PHP
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
common
core
demo
library
.gitignore
README.md
autoloader.php

README.md

Spider

phpSpider 使用phpSpider框架爬点东西 https://github.com/owner888/phpspider

IP池 可以查看 common下的iPool.php

demo文件夹内容:

jianshu_read.php 简书阅读爬虫

ipSpider.php ip爬取脚本

checIp.php ip新鲜度脚本

流程是: 先爬ip 入redis 然后刷阅读 easy!

crontab 命令 每10分钟跑一次 jianshu_read 每5分钟抓IP 每10分钟检测redis池中ip的新鲜度
*/10 * * * * php /home/wwwroot/spider/demo/jianshu_read.php >> /home/wwwlogs/jianshu.log

*/5 * * * * php /home/wwwroot/spider/demo/ipSpider.php >> /home/wwwlogs/ipSpider.log
*/10 * * * * php /home/wwwroot/spider/demo/checIp.php >> /home/wwwlogs/checkIp.log
You can’t perform that action at this time.