Web crawler that generates a sitemap given a base URL
-
Updated
Mar 15, 2020 - Go
Web crawler that generates a sitemap given a base URL
html-inspect extract general information about a html page.
smap is a site-mapping engine written in Go.
web crawler which fetches URLs and outputs crawl results to console as the crawl proceeds
GoShooter is a command-line interface (CLI) tool, written in Go, designed to take screenshots of websites provided in a list. It's a multi-threaded utility that allows custom timeout configurations for screenshot capture.
A very simple and flexible web crawler, support multiple run modes.
Starter project for web crawler with controller per site
A concurent web crawler library written in Go.
Crawl schedules and classes info from http://112.137.129.87/qldt/ and http://112.137.129.115/tkb/listbylist.php
Intro to Go example worked through in the Women Who Go Utah Workshop
Go application. Crawls the Steam store and collects Game records based on search criteria.
A web crawler to keep you updated about things you want to keep track of
Via Text Density Simple Web Crawler With Go
Add a description, image, and links to the web-crawler topic page so that developers can more easily learn about it.
To associate your repository with the web-crawler topic, visit your repo's landing page and select "manage topics."