HorizonSpider - Nodejs
The spider crawls each site in
siteList and submit crawled links to zeronet.
Then zeronet processes them, and downloads those zites.
It continues to traversal
siteList and crawls newly downloaded sites to get more links
Clone this repo and:
.env file and add configs as follows: