Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Use Avro to store all your values in HBase instead of regular columns

branch: avro-1.5

This branch is 0 commits ahead and 64 commits behind master

Fetching latest commit…

Octocat-spinner-32-eaf2f5

Cannot retrieve the latest commit at this time

Octocat-spinner-32 avrobase
Octocat-spinner-32 file
Octocat-spinner-32 handlersocket conversion to avro 1.5 April 11, 2011
Octocat-spinner-32 hbase
Octocat-spinner-32 memcached conversion to avro 1.5 April 11, 2011
Octocat-spinner-32 mysql
Octocat-spinner-32 redis conversion to avro 1.5 April 11, 2011
Octocat-spinner-32 sharded
Octocat-spinner-32 solr
Octocat-spinner-32 .gitignore add gitignore July 06, 2010
Octocat-spinner-32 LICENSE add license June 09, 2010
Octocat-spinner-32 README add a better README now that more people are looking at the system July 05, 2010
Octocat-spinner-32 pom.xml
README
The AvroBase interface attempts to abstract away the details of storing data within key/value stores by layering
a simple, broadly applicable, API on top of them along with an evolvable data description format for storing
the values within the key-value store. The API consists of a small number of verbs:

public interface AvroBase<T extends SpecificRecord, K> {
  Row<T, K> get(K row) throws AvroBaseException;
  K create(T value) throws AvroBaseException;
  void put(K row, T value) throws AvroBaseException;
  boolean put(K row, T value, long version) throws AvroBaseException;
  void delete(K row) throws AvroBaseException;
  Iterable<Row<T, K>> scan(K startRow, K stopRow) throws AvroBaseException;
  Iterable<Row<T, K>> search(String query, int start, int rows) throws AvroBaseException;
}

The AvroBaseException will wrap any underlying datastore issues that occur and is a runtime exception so will
play nicely beneath almost any API. The most complicated part of the system is the initial instantiation of an
AvroBase implementation. We are using Guice to bind parameters to the implementations. Here is an example taken
from the Memcached implementation:

  static class MABModule implements Module {
    static {
      String[] serverlist = {"localhost:11211"};

      SockIOPool pool = SockIOPool.getInstance();
      pool.setServers(serverlist);
      pool.initialize();
    }

    @Override
    public void configure(Binder binder) {
      binder.bind(MemCachedClient.class).toInstance(new MemCachedClient(true));
      binder.bind(String.class).annotatedWith(Names.named("schema")).toInstance("test_schema");
      binder.bind(String.class).annotatedWith(Names.named("table")).toInstance("test_user");
      binder.bind(String.class).annotatedWith(Names.named("family")).toInstance("profile");
    }
  }

  @Test
  public void testSave() throws AvroBaseException {
    AvroBase<User, String> userHAB = AvroBaseFactory.createAvroBase(new MABModule(), MAB.class, AvroFormat.BINARY);
    ...
  }

The AvroBaseFactory creates a child injector that also injects the AvroFormat into the implementation. This part of
the system is a work in progress and I am open to ideas on better ways to create and use the implementations.
Something went wrong with that request. Please try again.