Skip to content
Repology link checking daemon
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
ci
linkchecker
test
.travis.yml
COPYING
Makefile
README.md
hosts.yaml
repology-linkchecker.py

README.md

Repology link checker

Build Status Coverage Status

This is a standalone link validity checker daemon for Repology project.

It operates on links table in Repology PostgreSQL database, which contains homepage and download URLs of F/OSS projects extracted from packages by Repology, taking URLs from there, checking their validity and accessibility via both IPv4 and IPv6 and writes the status back.

Features

  • asyncio based, which means it doesn't require a lot of resources, but still can process a lot of links in parallel
  • it takes most care to not generate excess load on servers by maintaining per-host queues and adding delays between consequential requests to a single host
  • (TODO) checks link availability via both IPv4 and IPv6
  • (TODO) capable of FTP link checking

Author

License

GPLv3 or later, see COPYING.

You can’t perform that action at this time.