HTTP::Parser::Error: Could not parse data entirely (EDIT: with a fix) #27

thbar opened this Issue Apr 12, 2012 · 6 comments


None yet

3 participants


Tested against master today - when I do this to change what I track:

stream.options[:content] = new_content

I get this error (after 2 on_reconnect calls):

[GEM_ROOT]/gems/twitter-stream-0.1.14/lib/twitter/json_stream.rb:121:in `<<'
[GEM_ROOT]/gems/twitter-stream-0.1.14/lib/twitter/json_stream.rb:121:in `receive_data'
[GEM_ROOT]/gems/eventmachine-1.0.0.beta.4/lib/eventmachine.rb:179:in `run_machine'
[GEM_ROOT]/gems/eventmachine-1.0.0.beta.4/lib/eventmachine.rb:179:in `run'

I updated to master and it still applies.

Does anyone else meet this issue?


Just realized that the first line in the code I maintain was doing a bit of monkey patching:

module Twitter
  class JSONStream < EventMachine::Connection
    attr_reader :options

My guess is that the use case may not be currently supported by the gem, then. I'll dive more and see if I can make this work somehow.

Any hint welcome if you read this!

@thbar thbar added a commit to mremond/twitter-stream that referenced this issue Apr 16, 2012
@thbar thbar Fix for #27 - "Could not parse data entirely"
The parser needs to be reset in case of reconnection, or you'll get a

Figured out: on reconnect, the @parser also needs to be reset, otherwise it will attempt to continue parsing as defined by the previous connection.

I made a fix that works for me, although beware, I'm only discovering this gem so there will be a better way IMO.

How to use:

Signal.trap('USR1') do
  @should_restart = true

EventMachine::add_periodic_timer(1) do
  if @should_restart
    tracked = { |e| CGI.escape(e) }.join(',')
    stream.reconnect_with_updated_config do |options|
      options[:content] = "track=#{tracked}"
    @should_restart = false

Let me know if you find a better way to do this.


@mgreen @rud @abuiles if this stuff makes sense, let me know and I'll wrap a proper pull-request! (I suspect there's probably a built-in way to achieve that already, but unsure).


So how does one use this again? Just put in that code anywhere and it runs?


n'thing tibbon: can you provide some examples of a configuration or implementation of this fix or is this something that is going to be included in a release of the gem?


@tibbon see my sample above; basically you need a way to detect you need to change the keywords (for my customer's app here I used a signal because it was convenient to inform the program from outside), then use reconnect_with_updated_config. (all this after applying my patch at

Hope this helps!

@mremond mremond added a commit to processone/twitter-stream that referenced this issue Dec 22, 2013
@mremond mremond Fix for #27 - 'Could not parse data entirely'
The parser needs to be reset in case of reconnection, or you'll get a
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment