Switch branches/tags
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
107 lines (106 sloc) 9.99 KB
<!DOCTYPE html>
<meta content="text/html; charset=UTF-8" http-equiv="content-type">
<meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
<meta name="viewport" content="initial-scale=1, maximum-scale=1">
<link href=",700,900%7CRoboto+Mono" rel="stylesheet">
<style type="text/css">
body {
font-family: 'Roboto', sans-serif;
line-height: 1.6;
color: rgb(5,5,5);
background: rgb(255,255,255);
max-width: 60em;
header {
padding-bottom: 1em;
a {
color: darkslategrey;
a:hover {
color: darkslateblue;
<title>Web standards for content syndication</title>
<meta name="twitter:title" content="Web standards for content syndication">
<meta property="og:title" content="Web standards for content syndication">
<meta itemprop="name" content="Web standards for content syndication">
<meta property="og:url" content="/web-standards-for-content-syndication/">
<link rel="canonical" href="/web-standards-for-content-syndication/">
<meta property="og:type" content="article">
<meta property="article:published_time" content="2017-05-21">
<meta property="article:modified_time" content="2017-05-21">
<meta property="og:description" content="William Narmontas looks at different ways to synchronise web content back and forth between sites.">
<meta itemprop="description" content="William Narmontas looks at different ways to synchronise web content back and forth between sites.">
<meta name="description" content="William Narmontas looks at different ways to synchronise web content back and forth between sites.">
<meta name="twitter:description" content="William Narmontas looks at different ways to synchronise web content back and forth between sites.">
<meta property="og:site_name" content="Scala William">
<link rel="author" href="">
<link rel="publisher" href="">
<meta property="og:image" content="">
<meta itemprop="image" content="">
<meta name="twitter:image" content="">
<meta name="author" content="William Narmontas">
<meta name="twitter:card" content="summary">
<meta name="twitter:site" content="@ScalaWilliam">
<article class="h-entry">
<h1 class="p-name">Web standards for content syndication</h1>
<h2>By <a href="/" class="u-author">William Narmontas</a>, <a href="/web-standards-for-content-syndication/" class="u-url"><time class="dt-published" datetime="2017-05-21">May 21, 2017</time></a></h2>
<p>This page can be <a href="">edited on GitHub</a>.</p>
<p>Original article at <a href="" class="u-syndication">The Practical Dev</a>.</p>
<section class="e-content">
<p>I've been making websites since 2004 and the one thing that struck me is how brittle it is to synchronise content over HTTP and also how to do it instantaneously. For example, when I write a blog post I'd like it to be synchronised to my own website, perhaps posted to Twitter or get a trackback/pingback - many different targets with many different intentions.</p>
<p>Perhaps it's not a blog post but an automatically generated item. It may be urgent or non-urgent, it may be frequent or infrequent.</p>
<p>In this article, I'm only interested in one-way data sharing, so WebSockets is out of the picture.</p>
<p>Pulling by polling means initiating a request to a server every X minutes and then checking if the content has changed.</p>
<p>This introduces a maximum delay and requires setting up scheduled tasks. Also you want to make sure not to exceed usage limits of the target server.</p>
<p><a href="" target="_blank">Atom</a> and <a href="" target="_blank">RSS</a> are good for providing a structured pull approach.</p>
<p>Pushing means that as soon as your content is published, YOU call into different targets to notify them of what has happened. It's reactive.</p>
<p>There are three flavours of push. If you want to be receiving the pushes, you need to host a dynamic server which is a disadvantage especially if you use static hosting.</p>
<h3>Predefined set of targets</h3>
<p>If you wanted to tweet automatically, at the point of publishing, you'd call into the Twitter <a href="" target="_blank">silo API</a>, but now you have to customise it for every single target. This typically happens with an HTTP POST.</p>
<h3>Self-managing subscribers</h3>
<p>One interesting use case I have is in <a href="" target="_blank">ActionFPS</a> where I wish to automatically enter tournament data into Challonge when a "clan war" is played. One way for me to do it is by calling into Challonge directly (current, but brittle), and the other way is to create a separate service (akin to microservices) to react to these events.</p>
<p>But I wish other people to be able to react as well. So I'm going to try out <a href="" target="_blank">WebSub</a> which is a W3 Candidate Recommendation from 11 April 2017, previously known as <a href="" target="_blank">PubSubHubbub</a>.</p>
<p>This is what we should be doing in microservices; not tightly coupled approaches.</p>
<h3>Content-based targets</h3>
<p>... such as mentions. There's a new standard out there called Webmention, which is similar to the good old xml-rpc, pingback and trackback.</p>
<p>Webmention enables conversations. Here's the specification: <a href="" target="_blank"></a> - which is a W3C Recommendation as of 12 January 2017.</p>
<p>Effectively, if you want to receive webmentions, you place a <code>&lt;link rel="webmention"/&gt;</code> in your HTML and when someone mentions your page, they will call that endpoint.</p>
<p>Because of this approach, you can <a href="" target="_blank">use free services</a> like <a href="" target="_blank">Bridgy</a> (<a href="" target="_blank">open source</a>) to automatically post your content to Twitter and other places, and also syndicate responses <strong>back</strong>! You don't need centralisation or the "let's write our own Twitter adapter" approach above. Nice!</p>
<p>This is my favourite approach: EventSource (aka Server-Sent-Events). The W3 Draft is here: <a href="" target="_blank"></a> - it's also reactive.</p>
<p>EventSource is <a href="" target="_blank">available in most browsers except IE</a> - but you can use a <a href="" target="_blank">polyfill for IE</a>.</p>
<p>EventSource has a good <a href="" target="_blank">Node.js library</a> and also comes with reliability included - if you missed out an event then you can recover from the last event you saw. I made a <a href="" target="_blank">small contribution</a> to that library several weeks ago so that I could <a href="" target="_blank">synchronise logs from ActionFPS</a> to my local machine in real time. There's also a <a href="" target="_blank">Scala client for SSE</a>.</p>
<p>Advantage is that your receiver is a client and not a server, but a disadvantage is that it's a continuous connection.</p>
<h2>Push API</h2>
<p>Made for web browsers. <a href="" target="_blank">Push API: W3C Working Draft 08 May 2017</a>. Uses <a href="" target="_blank">service workers</a>.</p>
<h1>Further reading</h1>
<p>Of interest might also be:</p>
<li><a href="" target="_blank"></a></li>
<li><a href="" target="_blank"></a></li>
<li><a href="" target="_blank"></a></li>
<li><a href="" target="_blank"></a></li>
<li><a href="" target="_blank"></a></li>
<p>Basically, decide to use the approach purely based on the frequency and urgency of delivery of your content:</p>
<li>Infrequent &amp; not urgent, no server to receive - pull</li>
<li>Infrequent &amp; urgent, with server to receive - push</li>
<li>Urgent, browser-only - Push API</li>
<li>Frequent, no server to receive - pull-push</li>