GitHub - lionaneesh/Monitor-Web: Did you come across a website in which you were really interested, and wanted to check for any updates, or maybe you are interested in getting updates if the documentation of your favorite website was updated, Is the answer is yes, Monitor-Web is your one stop solution, Monitor-Web tracks any changes in your favorite content and alerts you with proper log of differences. So, now never waste time surfing the web to check if there are any updates, Simply add the website you wish to monitor and relax, Whenever you need to check simply run the program and it will automatically sync for any changes or are provide you with a diff like output. It works best for static websites, mainly online HTML ebooks, online documentation, course lists, wiki's or something similar.

lionaneesh / Monitor-Web Public

Notifications You must be signed in to change notification settings
Fork 16
Star 39

Did you come across a website in which you were really interested, and wanted to check for any updates, or maybe you are interested in getting updates if the documentation of your favorite website was updated, Is the answer is yes, Monitor-Web is your one stop solution, Monitor-Web tracks any changes in your favorite content and alerts you with …

View license

39 stars 16 forks Branches Tags Activity

Star

Notifications

Branches Tags

Name		Name	Last commit message	Last commit date
Latest commit History 22 Commits
LICENSE		LICENSE
README		README
crawler_config.py		crawler_config.py
crawler_db_handling.py		crawler_db_handling.py
crawler_log.py		crawler_log.py
sync.py		sync.py
website-list.txt		website-list.txt

Repository files navigation

Introduction
============

Did you come across a website in which you were really interested, and wanted to check for any updates, or maybe you are interested in getting updates if the documentation of your favorite library or code repository was updated, If the answer is yes, Monitor-Web is your one stop solution, Monitor-Web tracks any changes in your favorite content and alerts you with proper log of differences. So, now never waste time surfing the web to check if there are any updates, Simply add the website you wish to monitor and relax, Whenever you need to check simply run the program and it will automatically sync for any changes or are provide you with a diff like output. It works best for static websites, mainly online HTML ebooks, online documentation, course lists, wiki's or something similar.

Dependencies
============

Monitor-Web is written in python and follows a procedural structure. It uses some awesome libraries most of them are standard and one of them is third party.

The only third-party library it supports is BeautifulSoup, development site:-

    http://www.crummy.com/software/BeautifulSoup/

Usage
=====

To start Monitor-Web :-

Install BeautifulSoup.
Download or Clone the repository.
and run sync.py

Politeness
==========

Crawling a webpage repetitively can cause bandwith loss to the target site. It is recomended to sync data, atleast after a 15 minute interval.

Output
======

Monitor-Web outputs the diffs to stderr, which ofcourse can be redirected to the desired file. In UNIX it can be done in the following way :-

./sync.py 2> output.diff

Author
======

Aneesh Dogra (lionaneesh-at-gmail-dot-com)