Fetch in batches of 200

this should prevent morph running out of memory
everypolitician-scrapers · May 30, 2018 · 484f656 · 484f656
1 parent cf1d86f
commit 484f656
Showing 1 changed file with 1 addition and 1 deletion.
diff --git a/scraper.rb b/scraper.rb
@@ -34,4 +34,4 @@ def members
 morph_names = members.map { |w| w[:wikiname] }
 toget = morph_names | names.values.flatten.uniq
 
-EveryPolitician::Wikidata.scrape_wikidata(names: { en: toget })
+EveryPolitician::Wikidata.scrape_wikidata(names: { en: toget }, batch_size: 200)