Skip to content

Embark on an adventure into the depths of the web with this powerful web crawler made with Node, Axios and Cheerio.

License

Notifications You must be signed in to change notification settings

arnabpal2022/web-crawler

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

7 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

web-crawler

A Basic Web-Crawler Attempted in Node.js (Libraries used axios and cheerio). The Project demonstrates core web crawling functionalities:

  • fetching
  • parsing
  • link extraction
  • iterative link following
  • crawl limit control
  • success-error ratio calculation.

(working in progress)

About

Embark on an adventure into the depths of the web with this powerful web crawler made with Node, Axios and Cheerio.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published