Skip to content

Commit

Permalink
Save both decided and validated applications to the database.
Browse files Browse the repository at this point in the history
  • Loading branch information
Adrian McEwen committed Feb 15, 2020
1 parent 05a713e commit 98bd6b0
Showing 1 changed file with 6 additions and 1 deletion.
7 changes: 6 additions & 1 deletion scraper.rb
Expand Up @@ -6,9 +6,14 @@
auths.each_with_index do |auth, i|
begin
puts "#{i + 1} of #{auths.size}: Scraping #{auth.name}"
puts " Checking for validated applications..."
apps = auth.scrape({ validated_days: ENV['MORPH_DAYS'].to_i })
ScraperWiki.save_sqlite([:authority_name, :council_reference], apps)
puts " #{auth.name}: #{apps.size} application(s) saved."
puts " Now getting decided applications..."
apps = auth.scrape({ decided_days: ENV['MORPH_DAYS'].to_i })
ScraperWiki.save_sqlite([:authority_name, :council_reference], apps)
puts "#{auth.name}: #{apps.size} application(s) saved."
puts " #{auth.name}: #{apps.size} application(s) saved."
rescue StandardError => e
puts e
end
Expand Down

0 comments on commit 98bd6b0

Please sign in to comment.