mink is a command line SEO tool that allows you to crawl URLs and get their basic metrics including, but not limited to: HTTP status code, indexibility, emails, meta description, size of the page, number of internal and external links and others.
It is a simple command-line alternative to tools like Screaming Frog SEO Spider, Netspeak Spider and other. It is useful to create plain-text or CSV report that can be used in spreadsheet software for further analysis.
go get -u github.com/ribtoks/mink
Usage of mink: -d int Maximum depth for crawling (default 1) -f string Format of the output table|csv|tsv (default "table") -v Write verbose logs
mink reads URLs from
STDIN and writes reports to
STDOUT. Report can be written in a form of a table, comma-separated values and tab-separated values.
Crawl all pages of a single website:
echo "https://your-website.com" | mink -d 1000 -f csv > report.csv
Crawl a file with a list of URLs (1 per each line):
cat urls.txt | mink -f csv > report.csv