Lightweight library for scalable crawlers in Go.

this is a crawler to get http links that are related to a given webpage that you call when running the go run command

本工程基于golang开发脚手架https://github.com/wangshibiaoFlytiger/apiproject 创建, 功能: 爬取网络上的免费影片资源, 同时提供api接口, 输出爬取结果 除了用到脚手架内包含的技术, 还用到了goquery, 用于爬取静态页面

Crawl a Xenforo 2 thread by parsing raw html page source

Crawl javbus Magnet

Component to build broad crawls system

Super fast http crawler written in Go

Teach daily is web crawl by GoLang from web dev.to, freecodecamp.com, medium.com, hashnode.com, logrocket.com,infoq.com

web crawler

The Onion Crawler

gathertool是golang脚本化开发库，目的是提高对应场景程序开发的效率；轻量级爬虫库，接口测试&压力测试库，DB操作库等。

It allows you to crawl websites with various configurations, including crawl depth, timeout settings, proxy support, and output options.

Learn design pattern, crawl data

A fast and minimal JS endpoint extractor