Skip to content

App is a web scraper that utilizes the python tool Scrapy to scroll through Living Social, and saves local deals to a Postgres database. Source includes shell code to schedule the computer to run this script daily, so that rather than getting those annoying emails, users can query the database when they want a deal on sky diving or hot yoga.

Notifications You must be signed in to change notification settings

Ronnie619/Python-LivingSocial-Web-Scraper

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

15 Commits
 
 
 
 
 
 
 
 

Repository files navigation

Python-LivingSocial-Web-Scraper

App is a web scraper that utilizes the python tool Scrapy to scroll through Living Social, and saves local deals to a Postgres database. Source includes shell code to schedule the computer to run this script daily, so that rather than getting those annoying emails, users can query the database when they want a deal on sky diving or hot yoga.

To make a script that runs daily:

type crontab -e into the command line and add this line to the file:

0 13 * * * sh ~'location of your scrapy.sh file'

This says that ever day at hour 13 (1pm, relative to your local machine time), run the scrapy.sh script.

About

App is a web scraper that utilizes the python tool Scrapy to scroll through Living Social, and saves local deals to a Postgres database. Source includes shell code to schedule the computer to run this script daily, so that rather than getting those annoying emails, users can query the database when they want a deal on sky diving or hot yoga.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published