Skip to content
🐍 + 🤖 Python bot that crawls your website looking for dead stuff
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Type Name Latest commit message Commit time
Failed to load latest commit information.
LICENSE Fix example URL Aug 20, 2019

This was for my tutorial on building a dead link checker so its scope has been kept quite small.

Broken Link Crawler


Let's say I have a website and I want to find any dead links and images on this website.

$ python ''
> 404 -
> 404 -

It's that simple. The website is crawled, and all href and src attributes are sent a request. Errors are reported. This bot doesn't observe robots.txt but you should.

It is not a clever bot. But it is a good bot.

Accepting (small) PRs and issues!

You can’t perform that action at this time.