Skip to content
This repository


Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

client for tracking the lockstep protocol

This branch is 0 commits ahead and 0 commits behind master

Fetching latest commit…

Cannot retrieve the latest commit at this time


lockstep is a client for the lockstep protocol for keeping up to date on a set of changing data over HTTP.

lockstep protocol overview

Given an http endpoint, pull down records as a list of tuples expressed in json, one per line.

$ curl
{"id":123, "updated_at":999, "deleted_at":null, "ip":"", "port":1234}
{"id":124, "updated_at":1000, "deleted_at":1000, "ip":"", "port":1234}

The response can be finite, or be a chunked response that continues to feed updates to the receiver. The final element of the path endpoint is the most recent updated_at time seen and only updates equal to or after that time should be received.

All records sent must have an updated_at attribute to identify when they were last changed. Records should always be sent sorted by updated_at.

All records sent must have a deleted_at attribute which should be null normally, or the time at which the tuple was deleted. If the endpoint requests time '0' then deleted elements may be skipped.


$ make get-deps
$ make


start_link(Uri) -> Result
start_link(Uri, Opts) -> Result
  Uri = list() %% http endpoint to pull data from
  Opts = [Opt]
  Opt = {callback, Callback} | {table, TabName} | {ets_opts, EtsOpts} | {disk, SyncToDisk} | {order_by, OrderField}
  Callback = {Module, Function, Args}
  Module = atom()
  Function = atom()
  Args = list()
  TabName = atom() %% the name of the ets (and optionally dets) table to which lockstep data is written
  EtsOpts = list() %% arguments to pass to ets:new
  SyncToDisk = boolean() %% if the options list contains {disk, true}, lockstep will
                         %% sync its ets table to disk and pick up the stream from where
                         %% the dets table left off rather that resyncing the whole data set.
  OrderField = atom() %% specify the field to extract from json to use as the numeric sequence id
  Result = {ok, pid()} | {error, term()}


$ node server.js
Listening on port 4567

$ erl -pa ebin deps/*/ebin
1> {ok, Pid}  = lockstep:start_link("", [{table, servers}, {disk, true}]).
{ok, <0.33.0>
2> ets:tab2list(servers).
Something went wrong with that request. Please try again.