Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

A clone of Twitter's Snowflake, built on riak_core

branch: master

in Makefile, expect rebar to be on the PATH

The Makefile was trying to call a rebar executable as a sibling file to
itself in the top-level project directory, but no such executable
exists. Since rebar is a de facto Erlang build tool nowadays, just pick it
up from the PATH instead.
latest commit f4bb4907de
Steve Vinoski vinoski authored November 29, 2012
Octocat-spinner-32 apps update to use riak_core 1.2.1 November 29, 2012
Octocat-spinner-32 rel regenerate from revised rebar_riak_core templates for 1.0 October 06, 2011
Octocat-spinner-32 .gitignore regenerate from revised rebar_riak_core templates for 1.0 October 06, 2011
Octocat-spinner-32 LICENSE Add README and LICENSE. April 14, 2011
Octocat-spinner-32 Makefile in Makefile, expect rebar to be on the PATH November 29, 2012
Octocat-spinner-32 README.org Add a Webmachine resource so you can actually get those IDs. April 15, 2011
Octocat-spinner-32 rebar.config update to use riak_core 1.2.1 November 29, 2012
README.org

riak_id

Overview

riak_id is a very simple application that serves both to duplicate functionality from Twitter’s “Snowflake” and to teach, via a very simple example, how to build a riak_core application.

Design

The core functionality of riak_id is to provide k-sorted unique identifiers. The structure of the identifiers is very similar to Snowflake. They are 64-bit integers with the following layout (from highest to lowest):

Bits Description
1 Signedness flag, always 0
41 Unix timestamp, down to the millisecond
10 Top 10 bits of partition number (see Caveats)
12 Per-partition strictly increasing counter

Caveats

  • Decent NTP synchronization between machines is needed for riak_id to work as expected.
  • Request failures (e.g. from partition, timeout) should simply be retried. You will eventually get an ID.
  • Because only 10 bits are reserved for the partition number, there is a practical ring-size limit of 1024. This should not be a problem because the work is mostly CPU-bound.
  • So that we can give reasonable guarantees of uniqueness, some requests may have a minimum latency of 1ms (to allow the timestamp to increment).
  • It will run out of ID space within the century. Snowflake solves this using the “Twepoch”. I didn’t bother with that, but it would be a simple feature to add.
  • I wrote riak_id to learn, I hope it will be useful to others.

Quick Start

riak_id requires Erlang/OTP, rebar and a sane GNU build system.

  1. Clone the repository, copy the rebar script into the repository.
  2. Run make rel from the command line.
  3. rel/riak_id/bin/riak_id console
  4. riak_id:next_id(). from the console to generate IDs.

Acknowledgments

Thanks to Jesse Newland, Andy Gross, Ryan Zezeski for their work on the rebar template for riak_core apps, without which I would be lost. Special thanks to Andy for pointing out some egregious errors in an early version of the vnode code.

License

Copyright 2011, Sean Cribbs. Licensed under Apache License 2.0.

Something went wrong with that request. Please try again.