Skip to content

Commit

Permalink
Initial scraper
Browse files Browse the repository at this point in the history
  • Loading branch information
tmtmtmtm committed Aug 23, 2018
1 parent 35e8a67 commit 5175c2d
Showing 1 changed file with 21 additions and 0 deletions.
21 changes: 21 additions & 0 deletions scraper.rb
@@ -0,0 +1,21 @@
#!/bin/env ruby
# encoding: utf-8
# frozen_string_literal: true

require 'pry'
require 'scraperwiki'
require 'wikidata/fetcher'
require 'wikidata/area'

query = <<QUERY
SELECT DISTINCT ?item WHERE {
?item p:P31 [ps:P31 wd:Q192611; pq:P642 wd:Q4763974]
}
QUERY

wanted = EveryPolitician::Wikidata.sparql(query)
raise 'No ids' if wanted.empty?

ScraperWiki.sqliteexecute('DELETE FROM data') rescue nil
data = Wikidata::Areas.new(ids: wanted).data
ScraperWiki.save_sqlite(%i(id), data)

0 comments on commit 5175c2d

Please sign in to comment.