Skip to content

Commit

Permalink
Browse files Browse the repository at this point in the history
Restart the crawler every 15 minutes to work-around the synchronizati…
…on issues
  • Loading branch information
redox committed Oct 18, 2014
1 parent bcb7cde commit 650aee3
Show file tree
Hide file tree
Showing 2 changed files with 10 additions and 2 deletions.
2 changes: 1 addition & 1 deletion app/workers/hacker_news_realtime_crawler.rb
Expand Up @@ -6,7 +6,7 @@

class HackerNewsRealtimeCrawler

def cron
def crawler
EM.run do
@source = EventMachine::EventSource.new("#{ENV['HN_API_URL']}/v0/updates.json", headers = {"Accept" => "text/event-stream"})
@source.on "keep-alive" do |unused|
Expand Down
10 changes: 9 additions & 1 deletion bin/crawler
@@ -1,4 +1,12 @@
#!/usr/bin/env ruby

require File.expand_path(File.join(File.dirname(__FILE__), '..', 'config', 'environment'))
HackerNewsRealtimeCrawler.new.cron

begin
# work-arounds recent API hangs by restarting the crawler every 15 minutes
Timeout::timeout(60*15) do
HackerNewsRealtimeCrawler.new.crawler
end
rescue Timeout::Error
exit 1
end

0 comments on commit 650aee3

Please sign in to comment.