Skip to content
十年磨一剑:Crawler4U, a general purpose focused crawler
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
conf
rss
version
.gitignore
Makefile
README.md
clearenv.sh
main.go
parsers.go
start.sh
usage_cn.md
util.go
web.go

README.md

Crawler4U: A general purpose focused crawler

Overview

Crawler4U is a general purpose focused crawling and scraping tool based on json format configurations.

Install

The Golang way:

go get crawler.club/crawler

Or download the pre-built binaries from releases for you system.

Build from source

Before you can build the crawler from source. A workable golang development environment is needed. Downlad golang here and then execute the following commands.

go get -d crawler.club/crawler
cd $GOPATH/src/crawler.club/crawler
make

Usage

中文

Companies using crawler.club/crawler

You can’t perform that action at this time.