PubSubHubbub support

[ErisDS]

AKA PuSH or PubS'OhFuckIt

Despite being the worst name ever, PuSH is a pretty neat protocol for telling the world about new content. I'm really, really surprised there's not been more demand for support, or more tools that make use of PuSH over RSS, but I guess the name might be holding it back.

Nonetheless, we're a first class publishing platform, and I think PuSH support makes a lot of sense as a core offering of Ghost alongside our XMLRPC ping mechanism.

At this point, I haven't got / written a spec for how to implement this. After a cursory npm search there are some tools out there, but it may be that we should fork one of them and turn it into a canonical node module with full support for both hub and client behaviours.

There's a nice, detailed tutorial on creating a hub here: https://blog.superfeedr.com/howto-pubsubhubbub/

In order to be a hub, Ghost blogs will need to have a set of endpoints, keep records of subscribers and handle verification handshakes as well as pushing out content to the subscribers in the correct format.

At this point, I have not written a spec for what this should look like or how it should work. If anyone is interested in picking this up and running with it, I think it would be super cool to add.

[mike182uk]

Hey, i wouldn't mind implementing this - I've done a bit of work with the protocol before. I'll make a start and update this issue when i have a viable solution.

[ErisDS]

Looking forward to it! I think the first thing to figure out is whether each Ghost blog should really be a hub, or whether this is something that should hook into services like superfeedr, or a Ghost specific hub that we host somewhere.

In some senses, I think having hub support in each blog provides the more interesting approach - centralising things is always less preferable to giving people control. However on the flip side, it depends on the overhead of the feature - how much code is needed, etc etc, needs to balance out a bit with usage.

At this point, I think it's very much worth experimenting with though, as we can always move it out into an 'app' later when we've established that as an option.

[mike182uk]

@ErisDS Added a WIP PR for progress so far.

[ErisDS]

Hey @mike182uk, this is super cool! How does the experimentation you've done feed into my question above?

How do you feel about this implementation vs supporting 3rd party hubs?

P.S. Everything you've said about RSS makes sense - would love to pursue a refactor there as a separate piece of work. The current implementation doesn't feel right.

[mike182uk]

So, as you can see from the PR, there is quite a bit of code needed to get this in, especially around the hub side of things. But it is code that should rarely need to change, unless the PubSubHubbub spec is revised and we wanted to implement this.

My only real concern is sending notifications to the subscribers. Depending on the amount of the subscribers, and the amount of posts published, its possible that this could turn into quite an intensive process (i.e 5000 subscribers = 5000 HTTP requests but could be up to 50,000 HTTP requests if every subscriber failed to respond up to 10 times (configureable), although this is unlikely). I guess this is going to be proportional to the popularity of the blog though, so the more popular it is, the more resource is going to need anyway (which could negate the concern altogether).

If we was talking to an external hub, we would only need to send one HTTP request, and the hub would deal with updating the subscribers, taking away the above concern and also reducing the amount of code needed for this feature.

Sites like Techcrunch (hosted on Wordpress VIP) host their own hub using a WP plugin called PushPress. This basically does the same as what we are doing (but its possible they may have made modifications to the plugin with regards to how they notify subscribers).

Personally, with my the concern above put aside, i don't see why the Ghost blog shouldn't be its own hub, but... Using a third party hub is going to be the easier route, and also circumvents my concern above. The downside is that it locks you into using a centralised hub. The most popular open hub looks to be http://pubsubhubbub.appspot.com (google owned) which is used by FeedBurner, so i'd say its pretty robust.

One thought i have had though is it won't be an easy thing to switch from one method to another (i.e from using a self hosted hub to using a public hub or vice versa). Switching from self hosted to third party could be made possible by some kind of exporter that exports the current self hosted subscriptions into the new system. I don't think you could switch from third party to self hosted though without loosing your subscribers because AFAIK you can't export your subscribers from a public hub. The only exception to this would be if you had a Ghost specific centralised hub that had this functionality.

[ErisDS]

Going to close this issue now. It's over a year old, and in that year we haven't really had anyone ask for this. It only has 2 votes on our ideas board, and the implementation is sufficiently complex that we felt it was adding too much maintenance overhead to merge given how little this feature has been requested.