SEO: Add RPC Update Services for Faster Indexing

[JohnONolan]

At present, new posts published with Ghost are slightly slow to find their way into the search indexes. One way to accelerate this process is with the use of an XML-RPC to ping various services and let them know that a new post exists.

There are various extents to which this can be implemented. WordPress ships with a single service (pingomatic) - and mostly SEO-boffs ramp that up to 50+ services (overkill).

I would suggest that we implement a default, behind the scenes RPC connection which pings the 3 most popular services - any further modification to enable or disable more or less services should then be done via apps.

This is the list of what I believe is generally considered to be the top 3:

1. http://rpc.pingomatic.com
2. http://blogsearch.google.com/ping/RPC2
3. http://rpc.technorati.com/rpc/ping (looks dead, but I believe it actually only responds to POST requests)

We can review adding more by default later if it seems sensible, but 3 is probably a good starting point.

It's probably worth looking closely at how WordPress does this for reference. Not sure if pings should be sent only when a post is first published, or also when it is updated.

Would be great to get this into 0.5 if possible, but not essential.

//cc @halfdan

[halfdan]

Quickly searched for technorati ping: It seems they deactivated that in 2009 (http://technorati.com/ping).

[JohnONolan]

Top 2 it is then! Unless anyone suggests a 3rd place contender as a suitable replacement with reasons.

Also NB for this issue - pings should only be active when a blog is in the production environment. Should not be sent for dev or staging.

[halfdan]

halfdan.brainDump(): This is a great use-case for the upcoming post.published event. As for implementing the actual XML-RPC request I'm not sure whether it's worth pulling in another library (like node-xml) for it or simply create the request with a few xml strings.

[halfdan]

Just started implementing this:

• Ping-O-Matic doesn't bother returning a confirmation - so I have no idea if that service still works (or if my request is broken)
• Blogsearch always returns with a thank you regardless of what method you call. - herpDerp.blurp seems to be a totally valid ping method for them.
• Wordpress maintains a larger list: https://codex.wordpress.org/Update_Services#XML-RPC_Ping_Services
• Twingly (from the list) supports two methods - one normal ping and an extendedPing (with additional fields). http://rpc.twingly.com/

I am not sure whether I should validate the response (if any) from those services or just fire and forget. It's unclear what should happen when a blog ping fails.

[JohnONolan]

Nothing - I would suggest fire and forget. That RPC list on the codex is maintained by the general public btw, it's a wiki. So. Yeah.