Skip to content
This repository has been archived by the owner before Nov 9, 2022. It is now read-only.

pepijn/python-upsert-scraper

master
Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 
published

UPSERT scraper

Accompanying article: http://p.epij.nl/postgresql/heroku/2016/04/20/sidestepping-heroku-limits-with-postgresql-upsert/

Addons

$ heroku addons

Add-on                                         Plan       Price
─────────────────────────────────────────────  ─────────  ─────
heroku-postgresql (postgresql-graceful-65275)  hobby-dev  free
 └─ as DATABASE

scheduler (scheduler-trapezoidal-14878)        standard   free
 └─ as SCHEDULER

sendgrid (sendgrid-horizontal-56807)           starter    free
 └─ as SENDGRID

Heroku scheduler settings

Every 10 minutes:

curl -s "http://example.com" | python scraper.py --recipients you@example.com,friend@example.com --subject "Website changed"