Skip to content

Scraper to fetch, parse, and filter bird observations. Automated Telegram messaging functionality.

License

Notifications You must be signed in to change notification settings

joonarafael/tiirascraper

Repository files navigation

TIIRASCRAPER (PYTHON WEB SCRAPER)

main workflow codecov

Check the Installation Manual and the User Manual before advancing further. The user manual will provide you with the information on how to install dependencies, configure files for filtering, and initialize environment variables.

About

This is a simple web scraper software built with Python. Automated testing included as part of CI pipeline. Coverage report uploaded to Codecov.

It fetches the index page of a popular bird observation site Tiira and parses the latest most interesting bird observation records.

The program also enables the ability to create filters for individual cities and species, so that any records without matching criteria will be disregarded.

In addition, the program supports automated Telegram messaging feature. By configuring your own Telegram bot and applying relevant environment variables, you can get the latest records straight to your Telegram!

The program has been built to run "as a server"; it will automatically perform the procedure explained above every 5 minutes. It will read the configuration files and history again and check for a change on the site. The program also recovers from previous errors and e.g. unsuccessful connection attempts.