Grow your team on GitHub
GitHub is home to over 28 million developers working together. Join them to grow your own development teams, manage permissions, and collaborate on projects.Sign up
A little like that j-thing, only in Go.
Polite, slim and concurrent web crawler.
A simple and flexible web crawler that follows the robots.txt policies and crawl delays.
tiny Go library to normalize URLs
Package rehttp implements a Go HTTP transport that handles retries.
Proper URL escaping as per RFC3986