Skip to content


Subversion checkout URL

You can clone with
Download ZIP
client for tracking the lockstep protocol
Erlang JavaScript Perl Ruby
Pull request Compare This branch is 34 commits ahead, 1 commit behind orionz:master.

Fetching latest commit…

Cannot retrieve the latest commit at this time

Failed to load latest commit information.


lockstep is a client for the lockstep protocol for keeping up to date on a set of changing data over HTTP.

lockstep protocol overview

Given an http endpoint, pull down records as a list of tuples expressed in json, one per line.

$ curl
{"id":123, "updated_at":999, "deleted_at":null, "ip":"", "port":1234}
{"id":124, "updated_at":1000, "deleted_at":1000, "ip":"", "port":1234}

The response can be finite, or be a chunked response that continues to feed updates to the receiver. The final element of the path endpoint is the most recent updated_at time seen and only updates equal to or after that time should be received.

All records sent must have an updated_at attribute to identify when they were last changed. Records should always be sent sorted by updated_at.

All records sent must have a deleted_at attribute which should be null normally, or the time at which the tuple was deleted. If the endpoint requests time '0' then deleted elements may be skipped.


$ make get-deps
$ make


start_link(Uri) -> Result
start_link(Uri, Opts) -> Result
  Uri = list() %% http endpoint to pull data from
  Opts = [Opt]
  Opt = {callback, Callback} | {table, TabName} | {ets_opts, EtsOpts} | {disk, SyncToDisk} | {order_by, OrderField}
  Callback = {Module, Function, Args}
  Module = atom()
  Function = atom()
  Args = list()
  TabName = atom() %% the name of the ets (and optionally dets) table to which lockstep data is written
  EtsOpts = list() %% arguments to pass to ets:new
  SyncToDisk = boolean() %% if the options list contains {disk, true}, lockstep will
                         %% sync its ets table to disk and pick up the stream from where
                         %% the dets table left off rather that resyncing the whole data set.
  OrderField = atom() %% specify the field to extract from json to use as the numeric sequence id
  Result = {ok, pid()} | {error, term()}


$ node server.js
Listening on port 4567

$ erl -pa ebin deps/*/ebin
1> {ok, Pid}  = lockstep:start_link("", [{table, servers}, {disk, true}]).
{ok, <0.33.0>
2> ets:tab2list(servers).
Something went wrong with that request. Please try again.