Skip to content

Commit

Permalink
Get bills for both houses
Browse files Browse the repository at this point in the history
  • Loading branch information
henare committed Dec 13, 2014
1 parent 80e70ee commit 8a70ddc
Showing 1 changed file with 12 additions and 9 deletions.
21 changes: 12 additions & 9 deletions scraper.rb
Original file line number Diff line number Diff line change
Expand Up @@ -45,13 +45,16 @@ def get_bill(house, id)
end


house = "representatives"
bill_id = (ScraperWikiMorph.select("max(id) from data where house='#{house}'").first['max(id)'] || 1 rescue 1)

# Stop after 250 pages with no bill found. This might sound excessive
# but from what I can tell Representative bill IDs start at R240 and Senate at S111
while @bill_not_found_count <= 250
get_bill(house, bill_id)
bill_id += 1
["representatives", "senate"].each do |house|
bill_id = (ScraperWikiMorph.select("max(id) from data where house='#{house}'").first['max(id)'] || 1 rescue 1)
puts "*** Getting #{house} bills, starting at #{bill_id}"

# Stop after 250 pages with no bill found. This might sound excessive
# but from what I can tell Representative bill IDs start at R240 and Senate at S111
while @bill_not_found_count <= 250
get_bill(house, bill_id)
bill_id += 1
end
puts "*** Finished getting #{house} bills"
@bill_not_found_count = 0
end

0 comments on commit 8a70ddc

Please sign in to comment.