Smart JS Polling

While Comet may be all the rage, some of us are still stuck in web 2.0. And those of us that are use Ajax polling to see if there’s anything new on the server.

Here at GitHub we normally do this with memcached. The web browser polls a URL which checks a memcached key. If there’s no data cached, the request returns and polls again in a few seconds. If there is data, the request returns with it and the browser merrily goes about its business. On the other end our background workers stick the goods in memcached when they’re ready.

In this way we use memcached as a poor man’s message bus.

Yet there’s a problem with this: if after a few Ajax polls there’s no data, there probably won’t be for a while. Maybe the site is overloaded or the queue is backed up. In those circumstances the continued polling adds additional unwanted strain to the site. What to do?

The solution is to increment the amount of time you wait in between each poll. Really, it’s that simple. We wrote a little jQuery plugin to make this pattern even easier in our own JS. Here it is, from us to you:

Any time you see “Loading commit data…” or “Hardcore Archiving Action,” you’re seeing smart polling. Enjoy!