Development Setup How To

Paolo edited this page Aug 18, 2016 · 3 revisions
Clone this wiki locally

This document will guide you through each step required for having Hybris code set up for compiling and running it within Eclipse.

Development tools

Hybris is developed in Java 8 using Eclipse IDE.
We used Maven 3 as building tool and Git 1.8 for source version control.
Provided that you have installed all the tools mentioned above, you'll may still want to use the following Eclipse plugins: Egit and m2e. (NB: m2e is included in the Eclipse IDE for Java Developers package for versions equal or greater than 3.7 (Indigo))

For installing them:

  • on Eclipse EE: menu Help -> Eclipse Marketplace
  • on Eclipse Standard: Help -> Install New Software...

Getting the code

After having downloaded and installed the tools and Eclipse plugins listed above, import the project by downloading the code from the git repository hosted by BitBucket:

  1. File menu -> Import... -> Projects from git
  2. at the end of the wizard choose Import as a general project
  3. right click on the project -> Configure -> Convert to Maven project

ZooKeeper

Although having ZooKeeper running is not required by the unit tests included in the source code (thanks to the Apache Curator Test Server), at some point you'll want having it installed on your machine anyway.
To do so, you have to download the ZooKeeper distribution from its website and follow this brief tutorial to set a basic configuration and start it locally.

Alternately, Consul is also supported as metadata store.

Configuration setup

At this point you'll may want to set up Hybris configuration according to your likings, so let's see which are the two configuration files:

  • hybris.properties: includes the main Hybris settings
  • accounts.properties: includes the cloud storage accounts credentials and configurations

In particular, as a first step, in hybris.properties you'd need to fill the hybris.mds.address setting with the address of the Zookeeper server you've just started (e.g. "localhost:2181").
After that, you will need to fill the details about each of yours cloud storage accounts in accounts.properties, namely the access key (hybris.kvs.drivers.<account id>.akey) and the secret key (hybris.kvs.drivers.<account id>.skey) for each of them.
NB: the hybris.t parameter in hybris.properties sets the maximum number of tolerated failing clouds; so if you set it to 1, than you will need at least 2t+1, thus 3 cloud accounts to run Hybris.
Otherwise, for test sake, you may just want to use the transient fake in-memory key-value store instead, as in accounts-test.properties. In this case change the hybris.kvs.accountsfile property of hybris.properties to "accounts-test.properties".

Running the test CLI

Hopefully, at this point you should have a compiling copy of Hybris code along with your desired configuration.
To try Hybris run the simple command line interface testing client contained in fr.eurecom.hybris.test.HybrisTestCli.
With this client you'll be able to issue few commands that will let you store, delete, retrieve and list data on Hybris.

Erasure coding support

Erasure coding support is provided thanks to the jerasure library and its JNI bindings.
In the erasure coding parlance, k devices (clouds) are used for storing data blocks, while m are used instead for coding information. The original data is reconstructed using any k of the (k+m) stored blocks, so m devices (clouds) are allowed to fail. The size of the block stored in each cloud is about the original size divided by k.
k and m can be set respectively as hybris.erasurecoding.k and hybris.f.
To run Hybris with erasure coding support you should set as true the hybris.erasurecoding property and make sure that the jerasure library is compiled and linked to the project. For compiling the jerasure library shipped with Hybris just issue make from command line on the project root. For linking it to the project you must do one of the following:

  • set LD_LIBRARY_PATH=/path/to/jerasure/native/libs/ (In Eclipse: Run Configuration.. -> Environment tab)
  • or install both libraries (jerasure and jerasure.jni) into standard libs directories.

Git and Maven guides

Some useful links for getting started with Git and Maven: