Universal Binary Format
Erlang Python Java Shell
Pull request Compare This branch is 101 commits behind ubf:master.
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.


#The ubf application#

#Universal Binary Format#

Copyright (c) 2011 by Joseph Wayne Norton

Authors: Joseph Wayne Norton (norton@alum.mit.edu).

This is UBF, a framework for getting Erlang to talk to the outside world. This repository is based on Joe Armstrong's original UBF code with an MIT license file added to the distribution. Since then, a large number of enhancements and improvements have been added.

This repository is intended for production deployment and is deployed in carrier-grade systems.

Quick Start Recipe

To download, build, and test the ubf application in one shot, please follow this recipe:

$ mkdir working-directory-name
$ cd working-directory-name
$ git clone git://github.com/norton/ubf.git ubf
$ cd ubf
$ ./rebar get-deps
$ ./rebar clean
$ ./rebar compile
$ ./rebar eunit

For an alternative recipe with other "features" albeit more complex, please read further.


Where should I start?

This README is a good first step.

The UBF User's Guide is the best next step. Check out http://norton.github.com/ubf/ubf-user-guide.en.html for further detailed information.

One of the better places to start is to look in the "doc" directory. See the "Reference Documentation" section for suggestions on where to find greater detail.

The unit tests in the "test/unit" directory provide small examples of how to use all of the public API. In particular, the client.erl files contain comments at the top with a list of prerequisites and small examples, recipe-style, for starting each server and using the client.

The eunit tests in the "test/eunit" directory perform several smoke and error handling uses cases.

The #1 most frequently asked question is: "My term X fails contract Y, but I can't figure out why! This X is perfectly OK. What is going on?" See the the EDoc documentation for the contracts:checkType/3 function.

The documentation is in a state of slow improvement. Contributions from the wider world are welcome. :-)

What is UBF?

UBF is the "Universal Binary Format", designed and implemented by Joe Armstrong. UBF is a language for transporting and describing complex data structures across a network. It has three components:

  • UBF(a) is a "language neutral" data transport format, roughly equivalent to well-formed XML.

  • UBF(b) is a programming language for describing types in UBF(a) and protocols between clients and servers. This layer is typically called the "protocol contract". UBF(b) is roughly equivalent to Verified XML, XML-schemas, SOAP and WDSL.

  • UBF(c) is a meta-level protocol used between a UBF client and a UBF server.

See http://norton.github.com/ubf for further details.

What is EBF?

EBF is an implementation of UBF(b) but does not use UBF(a) for client and server communication. Instead, Erlang-style conventions are used instead:

  • Structured terms are serialized via the Erlang BIFs term_to_binary() and binary_to_term().

  • Terms are framed using the gen_tcp {packet, 4} format: a 32-bit unsigned integer (big-endian?) specifies packet length.

    | Packet length (32 bits) | Packet data (variable length) |

The name "EBF" is short for "Erlang Binary Format".

What about JSF and JSON-RPC?

See the ubf-jsonrpc open source repository https://github.com/norton/ubf-jsonrpc for details. ubf-jsonrpc is a framework for integrating UBF, JSF, and JSON-RPC.

What about TBF and Thrift?

See the ubf-thrift open source repository https://github.com/norton/ubf-thrift for details. ubf-thrift is a framework for integrating UBF, TBF, and Thrift.

What about ABNF?

See the ubf-abnf open source repository https://github.com/norton/ubf-abnf for details. ubf-abnf is a framework for integrating UBF and ABNF.

What about EEP8?

See the ubf-eep8 open source repository https://github.com/norton/ubf-eep8 for details. ubf-eep8 is a framework for integrating UBF and EEP8.

To download

  1. Configure your e-mail and name for Git

    $ git config \--global user.email "you@example.com"
    $ git config \--global user.name "Your Name"
  2. Install Repo

    $ mkdir -p ~/bin
    $ wget -O - https://github.com/android/tools_repo/raw/master/repo > ~/bin/repo
    $ perl -i.bak -pe 's!git://android.git.kernel.org/tools/repo.git!git://github.com/android/tools_repo.git!;' ~/bin/repo
    $ chmod a+x ~/bin/repo
    Caution Since access to kernel.org has been shutdown due to hackers, fetch and replace repo tool with android's GitHub repository mirror.
  3. Create working directory

    $ mkdir working-directory-name
    $ cd working-directory-name
    $ repo init -u git://github.com/norton/manifests.git -m ubf-default.xml
    Note Your "Git" identity is needed during the init step. Please enter the name and email of your GitHub account if you have one. Team members having read-write access are recommended to use "repo init -u git@github.com:norton/manifests.git -m ubf-default-rw.xml".
    Tip If you want to checkout the latest development version, please append " -b dev" to the repo init command.
  4. Download Git repositories

    $ cd working-directory-name
    $ repo sync

For futher information and help for related tools, please refer to the following links:

To build - basic recipe

  1. Get and install an erlang system http://www.erlang.org

  2. Build

    $ cd working-directory-name/src
    $ make compile
  3. Run the unit tests

    $ cd working-directory-name/src
    $ make eunit

To build - optional features

  1. Dialyzer Testing basic recipe

    1. Build Dialyzer's PLT (required once)

      $ cd working-directory-name/src
      $ make build-plt
      Tip Check Makefile and dialyzer's documentation for further information.
    2. Dialyze with specs

      $ cd working-directory-name/src
      $ make dialyze
      Caution If you manually run dialyzer with the "-r" option, execute "make clean compile" first to avoid finding duplicate beam files underneath rebar's .eunit directory. Check Makefile for further information.
    3. Dialyze without specs

      $ cd working-directory-name/src
      $ make dialyze-nospec
  2. To build the Java client and run its encoding/decoding unit test:

    $ cd working-directory-name/src
    $ make -C lib/ubf/priv/java
  3. The Python client depends on the "py-interface" library. To clone and build it, use:

    $ cd working-directory-name
    $ git clone git://repo.or.cz/py_interface.git
    $ cd py_interface
    $ autoconf
    $ make

    Then install as a normal Python package or run using "env PYTHONPATH=working-directory-name/py_interface python your-script.py"


  • QuickCheck/Proper tests

  • Add WebSockets support

  • Add more Thrift (http://incubator.apache.org/thrift/) support

    • Compact Format

  • Add Google's Protocol Buffers (http://code.google.com/apis/protocolbuffers/) support

  • Add Bert-RPC (http://bert-rpc.org/) support

    • BERT-RPC is UBF/EBF with a specialized contract and plugin handler implementation for BERT-RPC. UBF/EBF already supports all of the BERT data types. UBF is the text-based wire protocol. EBF is the binary-based wire protocol (based on Erlang's binary serialization format).

  • Add multiple listeners for a single UBF server support

  • Add UDP transport support


Many, many thanks to Joe Armstrong, UBF's designer and original implementor.

Gemini Mobile Technologies, Inc. has approved the release of its extensions, improvements, etc. under an MIT license. Joe Armstrong has also given his blessing to Gemini's license choice.