This is a crawler I made while learning Node Js,It have some bugs..
Node version > 8.0
npm install (execute this command inside the NodeJs-crawler directory)
- Change the target url in the source code currently it is https://google.com
command:- node app.js