Userstream not working anymore.

[DarkDefender]

I'm having problems with user streams since a few days back.
I can use the example in the README file and it seems to connect fine.

But I don't get any messages what so ever.. And the stream connection seems to still be alive. I seems to be alive because even if I set the socket timeout to be about 1 min it never times out.

It doesn't seem to raise any errors either so I don't really know what to do now.

"Normal" streams IE just TwitterStream without specifying domain works so I have no Idea what the problem could be.

[Telofy]

Same here. The problem started 2013-11-23 for one app and three days later for another one. I thought it was some sort of isolated outage maybe related to my apps or recent change in client IP and reported it here, but that’s getting less likely with every day I can’t find any of the other countless users of the Twitter API reporting similar problems. (Except in this thread.)

The sample stream works for me as well.

[Telofy]

When I insert a raise in this except block I get the following, because utf8_buf is an empty string (despite the name it’s a Unicode string by the way). Not sure what to do about this though.

In [4]: for tweet in twitter.user(**{'with': 'followings'}):
    print tweet
   ...:     
u''
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-4-8179b75ca0cb> in <module>()
----> 1 for tweet in twitter.user(**{'with': 'followings'}):
      2     print tweet
      3 

/home/telofy/documents/workspace/pony-news-rt/eggs/twitter-1.10.2-py2.7.egg/twitter/stream.py in __iter__(self)
     36                 utf8_buf = self.buf.decode('utf8').lstrip()
     37                 print repr(utf8_buf)
---> 38                 res, ptr = self.decoder.raw_decode(utf8_buf)
     39                 self.buf = utf8_buf[ptr:].encode('utf8')
     40                 yield wrap_response(res, self.handle.headers)

/usr/lib/python2.7/json/decoder.pyc in raw_decode(self, s, idx)
    381             obj, end = self.scan_once(s, idx)
    382         except StopIteration:
--> 383             raise ValueError("No JSON object could be decoded")
    384         return obj, end

ValueError: No JSON object could be decoded

[Telofy]

It’s also using 100% of one of my cores, which may be related to this fix. I hacked in a one-second sleep for now until this is fixed.

[DarkDefender]

Glad I'm not the only one with problems.
Perhaps we should open a bug ticket at: https://dev.twitter.com/issues
Because right now we don't get any indication on what could be wrong from the twitter stream.

I mean if we are doing something wrong when connecting to it, it should at least disconnect us.
Now it just keeps the connection alive and nothing is sent.

[Telofy]

The only reports of problems that I can find are here. If it were a general problem with the Twitter API, the Internet would be flooded with reports.

I also sniffed my connection, and while it was encrypted of course, so that I couldn’t see the package content, I could reliably trigger bigger packages by listening to the stream and then favoriting and unfavoriting tweets with the respective account. That also indicates that the problem is not solely with the API (unless it was some crazy degree of coincidence paired with my human bias to see patterns in random noise).

[DarkDefender]

I don't think it would hurt to report the problem. The worst that could happen is that they close the ticket as invalid.
The problem is really weird because it only seems to be the userstream that has this problem. If it were a error in how the twitter lib reads the tweets then I don't think any stream type should be working.

[agates]

Hi, I've encountered this problem as well. I had thought it was a problem with my experimental code until I realized the socket isn't receiving data at all. Not really sure what changed.

[Telofy]

I hope this will get fixed eventually. This library has by far the sanest API I’ve seen. The easiest way is probably to delegate all the HTTP-related functionality to requests.

All this low-level socket-listening is yucky to me, so I haven’t figured out what exactly the problem is, but I’ve created a fork of Twython where I have replaced their awkward streaming contraption with a yield (pretty much just like this library does it). There may’ve been advantages to their error handling, but I removed that as well for consistency, and so you’re no longer forced to subclass the streamer to use it. (Once this is fixed, I’ll switch back so I’ll no longer have to maintain the fork.)

Careful, by the way, Twython expects the authentication parameters in the order consumer_key, consumer_secret, oauth_token, oauth_secret.

[homlett]

Hi, I've this problem too with my app. It's very embarrassing and I'm surprise nobody knows where the problem coming from. I hope a solution... Regards.

[pauldebeurs]

I also have problems with getting tweets. Until november 2013 these statements worked:
...
iterator = twitter_stream.statuses.filter(track=track)
for tweet in iterator:
print tweet['id']
...
Since november 2013 the line 'for tweet in iterator:' never gives an answer. I get nothing, not even an error message!
I recently updated to twitter 1.10.2 (November 20, 2013), but that does not helped.
On https://blog.twitter.com/2013/rest-api-ssl-certificate-updates (Tuesday, November 19, 2013) I read: This means that all HTTP clients used by your application must trust the new root certificate, otherwise you won’t be able to connect in the API.
Just bad luck, this timing of the twitter 1.10.2 version?

[RouxRC]

@pauldebeurs 2 different pull requests propose fixes you can use by switching to one of their respective branches:
#196
#195

[Telofy]

Oh, some good work there. Thank you both! <3

[DarkDefender]

This is great! I hope the changes gets merged soon.

[sixohsix]

Sorry for the delay. PR #196 is merged, and 1.11.0 is released with the fix. Big thanks to the contributors who did all the work on this! (It totally wasn't me.)