A simple crawler that uses the Breadth First Search algorithm to crawl a given website and its sup-pages. It reports any links that lead to websites that are not available (i.e. 'broken links'). I wrote a similar crawler around 4 years ago using PHP. I added it to the repository for comparison.
-
Notifications
You must be signed in to change notification settings - Fork 0
PJUllrich/Link-Crawler
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
About
Python Crawler that reports broken links on a given website and its sup-pages
Topics
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published