Skip to content


Subversion checkout URL

You can clone with
Download ZIP
Use Avro to store all your values in HBase instead of regular columns
Latest commit c5b55c3 @spullara go deeper
Failed to load latest commit information.
avrobase nothing to see here
caching revert that changeg
file better reporting for a bad schema
gae add the google app engine libraries
handlersocket conversion to avro 1.5
hbase move timestamp generator to the main system
memcached conversion to avro 1.5
mongodb initial version of mongodb
mysql close statements and result sets properly
s3archive delete the temp files after uploading
sharded caching layer for avrobase
solr go deeper
.gitignore add gitignore
LICENSE add license
README add a better README now that more people are looking at the system
pom.xml initial version of mongodb


The AvroBase interface attempts to abstract away the details of storing data within key/value stores by layering
a simple, broadly applicable, API on top of them along with an evolvable data description format for storing
the values within the key-value store. The API consists of a small number of verbs:

public interface AvroBase<T extends SpecificRecord, K> {
  Row<T, K> get(K row) throws AvroBaseException;
  K create(T value) throws AvroBaseException;
  void put(K row, T value) throws AvroBaseException;
  boolean put(K row, T value, long version) throws AvroBaseException;
  void delete(K row) throws AvroBaseException;
  Iterable<Row<T, K>> scan(K startRow, K stopRow) throws AvroBaseException;
  Iterable<Row<T, K>> search(String query, int start, int rows) throws AvroBaseException;

The AvroBaseException will wrap any underlying datastore issues that occur and is a runtime exception so will
play nicely beneath almost any API. The most complicated part of the system is the initial instantiation of an
AvroBase implementation. We are using Guice to bind parameters to the implementations. Here is an example taken
from the Memcached implementation:

  static class MABModule implements Module {
    static {
      String[] serverlist = {"localhost:11211"};

      SockIOPool pool = SockIOPool.getInstance();

    public void configure(Binder binder) {
      binder.bind(MemCachedClient.class).toInstance(new MemCachedClient(true));

  public void testSave() throws AvroBaseException {
    AvroBase<User, String> userHAB = AvroBaseFactory.createAvroBase(new MABModule(), MAB.class, AvroFormat.BINARY);

The AvroBaseFactory creates a child injector that also injects the AvroFormat into the implementation. This part of
the system is a work in progress and I am open to ideas on better ways to create and use the implementations.
Something went wrong with that request. Please try again.