Crawler4U: A general purpose focused crawler
Crawler4U is a general purpose focused crawling and scraping tool based on json format configurations.
The Golang way:
go get crawler.club/crawler
Or download the pre-built binaries from releases for you system.
Build from source
Before you can build the
crawler from source. A workable golang development environment is needed. Downlad golang here and then execute the following commands.
go get -d crawler.club/crawler cd $GOPATH/src/crawler.club/crawler make