新一代爬虫平台，以图形化方式定义爬虫流程，不写代码即可完成爬虫。

Open-source Enterprise Grade Search Engine Software

Bot para monitoramento de promoções no fórum do Hardmob http://www.hardmob.com.br/promocoes/

Java web crawling library

A ZAPROXY Add-on that allows testing of web application vulnerabilities by recording complex multi-step sequences. You can test applications that need to access pages in a specific order, such as shopping carts or registration of member information.

Sample MVP project uses jsoup-web-crawl like API

A simple ZhiHu Crawler using WebMagic

A Library for web crawling websites harvesting URLs of embedded links and images

An async web crawler for ads.txt project

A web crawler that implements breadth first search algorithm and built with maven.

spring-boot-webcrawler-rest-demo-using-completable-futures

🔍 A web crawling app written in java.

This project provides a REST API that allows users to submit URLs for crawling. The app internally uses RabbitMQ to publish the URLs, and then listens back to fetch the contents of the URLs using Jsoup. The app also scrapes links and indexes the content using Apache Lucene.

自动爬虫

Cosmos is a WebCrawler + SearchEngine written in Java

Web-crawler - fetches available links from the given website and provides keyword search

A simple Web Crawler built with Spring Boot.

Command line interface for Frog Fronts Web Crawler library.