一个基于webmagic框架二次开发的java爬虫框架实战,已实现能爬取腾讯,搜狐,今日头条(单独集成功能)等资讯内容,配合elasticsearch框架用法,实现了自动爬虫,已投入线上生产使用。
-
Updated
Nov 16, 2022 - Java
一个基于webmagic框架二次开发的java爬虫框架实战,已实现能爬取腾讯,搜狐,今日头条(单独集成功能)等资讯内容,配合elasticsearch框架用法,实现了自动爬虫,已投入线上生产使用。
天气爬虫(全国城镇天气自动定时抓取更新,并开放RESTful查询接口),附带代理IP池定时更新并检测其可用性
SpringBoot+Solr + webmagic JD商品爬取数据,放入solr中做搜索,学习下solr使用
网络数据采集技术—Java网络爬虫 (书稿完整代码,涉及网络爬虫的各种技术和知识点)
基于springboot+mybatis+echarts+webmagic 的疫情数据可视化网站
抓取twitter数据,可根据时间、话题、用户名等条件抓取数据,twitter爬虫
🎉基于Springboot的SSM脚手架,目前已整合spring-scurity,websocket,docker,echarts,mybatis,elsticSearch.logback,ehcache,redis,kafka,jwt等,旨在开箱即用,简化搭建流程.集成了爬虫项目,OpenCV项目.WebSocket项目.
A dynamic crawler plug-in for the Android platform based on Dex dynamic loading, which can dynamically load and execute the dex plug-in package, and can realize real-time updates of crawler and other functions.
Netease cloud music spider, realized with webmagic which is a spider framework written by java.
Add a description, image, and links to the webmagic topic page so that developers can more easily learn about it.
To associate your repository with the webmagic topic, visit your repo's landing page and select "manage topics."