My First Python Test Script utilizing MySQLdb as database connector to extract and manipulate webs sites data.
The purpose of this script is to build an app which satisfies the following requirements:-
- read a list of URL's (CSV) provided in the the attached file, urls.txt and a list of tags (CSV) provided in the attached file tags.txt. Use appropriate data structures, like dictionaries tuples etc.
- fetch the page the url points to (in string format.)
- search for the 'tags' in each page (can use regex or other string operations) and count their occurrence.
- store the frequency count of each tag in a mysql DB.
ALERT: This is not just about making the program run and get things done, make it efficient and keep in mind future expandability. It should be objectified, documented and handle errors. In short, make it beautiful :)
This will help beginners cover the basics of programming, python syntax, data structures, loops and some basic libraries like urllib, urllib2, mysqldb, re etc.