Use Avro to store all your values in HBase instead of regular columns
Java
Latest commit c5b55c3 Jun 24, 2011 @spullara go deeper
Permalink
Failed to load latest commit information.
avrobase nothing to see here May 13, 2011
caching revert that changeg May 18, 2011
file better reporting for a bad schema Jun 5, 2011
gae add the google app engine libraries May 11, 2011
handlersocket
hbase move timestamp generator to the main system Apr 26, 2011
memcached conversion to avro 1.5 Apr 11, 2011
mongodb initial version of mongodb Jun 5, 2011
mysql close statements and result sets properly Jun 19, 2011
redis conversion to avro 1.5 Apr 11, 2011
s3archive
sharded caching layer for avrobase May 10, 2011
solr go deeper Jun 25, 2011
.gitignore add gitignore Jul 6, 2010
LICENSE add license Jun 9, 2010
README add a better README now that more people are looking at the system Jul 5, 2010
pom.xml

README

The AvroBase interface attempts to abstract away the details of storing data within key/value stores by layering
a simple, broadly applicable, API on top of them along with an evolvable data description format for storing
the values within the key-value store. The API consists of a small number of verbs:

public interface AvroBase<T extends SpecificRecord, K> {
  Row<T, K> get(K row) throws AvroBaseException;
  K create(T value) throws AvroBaseException;
  void put(K row, T value) throws AvroBaseException;
  boolean put(K row, T value, long version) throws AvroBaseException;
  void delete(K row) throws AvroBaseException;
  Iterable<Row<T, K>> scan(K startRow, K stopRow) throws AvroBaseException;
  Iterable<Row<T, K>> search(String query, int start, int rows) throws AvroBaseException;
}

The AvroBaseException will wrap any underlying datastore issues that occur and is a runtime exception so will
play nicely beneath almost any API. The most complicated part of the system is the initial instantiation of an
AvroBase implementation. We are using Guice to bind parameters to the implementations. Here is an example taken
from the Memcached implementation:

  static class MABModule implements Module {
    static {
      String[] serverlist = {"localhost:11211"};

      SockIOPool pool = SockIOPool.getInstance();
      pool.setServers(serverlist);
      pool.initialize();
    }

    @Override
    public void configure(Binder binder) {
      binder.bind(MemCachedClient.class).toInstance(new MemCachedClient(true));
      binder.bind(String.class).annotatedWith(Names.named("schema")).toInstance("test_schema");
      binder.bind(String.class).annotatedWith(Names.named("table")).toInstance("test_user");
      binder.bind(String.class).annotatedWith(Names.named("family")).toInstance("profile");
    }
  }

  @Test
  public void testSave() throws AvroBaseException {
    AvroBase<User, String> userHAB = AvroBaseFactory.createAvroBase(new MABModule(), MAB.class, AvroFormat.BINARY);
    ...
  }

The AvroBaseFactory creates a child injector that also injects the AvroFormat into the implementation. This part of
the system is a work in progress and I am open to ideas on better ways to create and use the implementations.