Streaming Updates for v6

[stve]

Just getting started but wanted to enumerate the priorities. I'm sure I'm missing a few but here's some things that I think should be on the punchlist:

• exponential back-off reconnects as defined under Reconnecting section of Twitter docs
• site stream controls
• update and reconnect (see Steaming API - updating filters/follows #565)
• 100% c0 code coverage for streaming
• additional message support (see Missing status responses for Twitter::Streaming::Client #649)

[sferik]

@stve This looks like a good start and a solid checklist.

I would add one item:

• 100% c0 code coverage for streaming

Currently, this library has 100% code coverage, except for one method: Twitter::Streaming::Connection#stream, which is faked in the Twitter::Streaming::Client specs. I was okay with less-than-complete code coverage when this was an experimental feature, but with the experimental warning label coming off in version 6, everything ought to be properly tested.

As always, let me know how I can be helpful. That said, I consider you to be the domain expert when it comes to Twitter Streaming, given your experience designing and maintaining em-twitter and TweetStream. If I had a clear idea of how to implement the items on the list above, I probably would have done it by now. I guess that’s another way of saying thanks so much for your help on this. I couldn’t do it without you. 😄

[stve]

@sferik totally agree on test coverage. The changes I just committed brought coverage on Twitter::Streaming::Connection up to 87.88% (and overall coverage to 99.78%) so we are getting closer.

[sferik]

@stve That’s great news!

[sferik]

I rebased master (:scream:) leading up to the 5.15.0 release because I wanted to get some of the non-6.0.0-specific features and bug fixes out into the world (see the CHANGELOG for details), since it had been more than 6 months since the previous release. I was hoping 6.0.0 would be released by now but it’s been slower going than expected for a variety of reasons, many of which are out of our control (e.g. blockers in the http gem and extremely high churn in the Twitter Ads API). That means that 6.0.0 is probably not coming any time soon, unless we dramatically start cutting scope.

In the best-case scenario, I’ll get a large chunk of dedicated time to work on this library in December and we can ship around the New Year. Worst-case is probably mid-2016, at which point, I’ll agressively start cutting scope just to ship whatever’s ready and start working on adding non-breaking changes in 6.1.0 and breaking changes in 7.0.0.

The good news is there is a clean diff between the 5-stable and master branches: master is based on 5-stable and includes only the changes that are specific to version 6. The bad new is all other branches based on master need to be rebased. This will be a relatively painful rebase (~50 commits). I apologize in advance for making you go through that, but I think it will be worth having a clean separation between the branches as we push toward a 6.0.0 release.