`sysread': end of file reached (EOFError)

[mrzacharycook]

I'm encountering the error below after typically < 2 hours of run time. I've encountered this with Ruby 1.9.3, and Ruby 2.1.0, installing the twitter gem with "gem install twitter":

/home/pi/.rbenv/versions/2.1.0/lib/ruby/2.1.0/openssl/buffering.rb:129:in 'sysread': end of file reached (EOFError) from /home/pi/.rbenv/versions/2.1.0/lib/ruby/2.1.0/openssl/buffering.rb:129:in 'readpartial' from /home/pi/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/twitter-5.7.1/lib/twitter/streaming/connection.rb:14:in 'stream' from /home/pi/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/twitter-5.7.1/lib/twitter/streaming/client.rb:111:in 'request' from /home/pi/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/twitter-5.7.1/lib/twitter/streaming/client.rb:53:in 'sample' from testtwitter.rb:24:in 'get_tweet_colors' from testtwitter.rb:37:in '<main>'

The error appears to be occurring doing the loading of an object at "@client.sample do |object|". The code I used to generate this error is:

require 'twitter'
require 'json'

class ColorGetter
  #Initialize the ColorGetter with an amount of tweets to process and the brightness adjustment value
  def initialize
    #Read the config file
    json_config = JSON.parse(File.read('twitter_config.json'))

    #then create a new streaming Twitter object.
    @client = Twitter::Streaming::Client.new do |config|
      config.consumer_key = json_config['consumer_key']
      config.consumer_secret = json_config['consumer_secret']
      config.access_token = json_config['access_token']
      config.access_token_secret = json_config['access_token_secret']
    end
  end

  #Call get_tweet_colors
  def get_tweet_colors
      puts 'starting sample'
      count = 0
      debug_object = nil
      @client.sample do |object|
        debug_object = object
        puts object.class.to_s
        count += 1
        puts count
      end
      puts 'exiting getter loop'
      puts debug_object
  end
end


bob = ColorGetter.new
bob.get_tweet_colors

Using a library like TweetStream, I've been able to run without error. Any thoughts why this is breaking?

[sferik]

@spagalloco Any idea why this might be happening?

[stve]

It sounds like you are experiencing an occasional disconnect. @sferik, it looks like we need to add reconnection logic into streaming connections. This gets fairly hairy as Twitter requires exponential back-offs, em-twitter's implementation is primarily here: https://github.com/tweetstream/em-twitter/blob/master/lib/em-twitter/connection.rb#L254-L293

I'm happy to help with this. On em-twitter, I tried to start simple, but this quickly grew fairly complicated.

@howpeegoo, for now, you could rescue EOFError and perhaps implement some simple retry logic paired with calls to sleep to reconnect without getting blocked by Twitter:

def get_tweet_colors
  tries = 5

  puts 'starting sample'
  count = 0
  debug_object = nil
  @client.sample do |object|
    debug_object = object
    puts object.class.to_s
    count += 1
    puts count
  end
  puts 'exiting getter loop'
  puts debug_object

  rescue EOFError => 
    if (tries -= 1) > 0
      sleep 10
      retry
    else
      raise e
    end
  end
end

[sferik]

@stve Any interest in attempting to add reconnection handling for streaming to the twitter gem? I’m going through old issues, working toward the next major release. Although this feature might not necessitate a major version bump, it would be a nice headline feature to include in 6.0.0.

[stve]

@sferik definitely, would be a great addition to 6.0.0