Skip to content
This repository has been archived by the owner on May 20, 2021. It is now read-only.

Commit

Permalink
Automatic update to make ScraperWiki scraper work on Morph
Browse files Browse the repository at this point in the history
  • Loading branch information
mlandauer committed Jan 22, 2014
1 parent 3fdd24a commit 680bbca
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion scraper.rb
@@ -1,3 +1,4 @@
require 'scraperwiki'
require 'mechanize'

url = "http://www.vincent.wa.gov.au/Your_Community/Whats_On/Community_Consultation/Planning_Applications"
Expand All @@ -15,7 +16,7 @@
'date_scraped' => Date.today.to_s,
'on_notice_to' => i.at('.listitemtext > b').next_sibling.inner_text.split('/').reverse.join('-')
}
if (ScraperWiki.select("* from swdata where `council_reference`='#{record['council_reference']}'").empty? rescue true)
if (ScraperWiki.select("* from data where `council_reference`='#{record['council_reference']}'").empty? rescue true)
ScraperWiki.save_sqlite(['council_reference'], record)
else
puts "Skipping already saved record " + record['council_reference']
Expand Down

0 comments on commit 680bbca

Please sign in to comment.