eyecare

Code to do some analysis for eye care

Build the code

make build-go

You can use Google Search to find sites to crawl. This uses Nimbleway's SERP API so you will need a Nimbleway account.

build/eyecare --outfile=results/results.jsonl --query="optometrist in san mateo"

To manually add a domain to the list

build/eyecare domains add https://pacificaeyedoctor.com/

To scrape all the sites listed in the JSONL file.

build/eyecare scrape --input=results/results.jsonl --output=~/scraped_sites

This will scrape the sites and dump the output in the directory specified as the output argument.

Dashes in URL names are replaced with underscores in the file.

e.g. the file clearvision1.com/about_me.unknown corresponds to the URL clearvision1.com/about-me