LStore - A fault-tolerant, performant distributed data storage framework.
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
binding Properly set the *_FOUND variables for the Globus libs Oct 5, 2016
build Remove gitignores (#42) Apr 15, 2016
cmake Purge some more ACCRE APR references Mar 21, 2017
debian Clean all compiler warnings Jun 20, 2016
doc Add reference counting implementation Jul 8, 2016
scripts Add slave script to build slave Feb 24, 2018
src Fix cleanup bug Apr 25, 2017
test Fix ZMQ bugs and add DNS TCP references. Apr 25, 2017
vendor/jerasure
.dockerignore
.gitignore Re-implement versioning scheme within CMake Aug 3, 2016
CMakeLists.txt Purge some more ACCRE APR references Mar 21, 2017
Jenkinsfile Bump LTS version for building Feb 24, 2018
LICENSE.txt Update License. (#75) May 3, 2016
README.md
VERSION Re-implement versioning scheme within CMake Aug 3, 2016
lstore.spec Re-implement versioning scheme within CMake Aug 3, 2016
sort.py Add sorting code May 25, 2016

README.md

LStore Release Tools

Structure

  • binding - Bindings to third-party applications
  • build - Location where all sources are built
    • logs - Build logs
    • package - Storage with built RPMs
    • repo - Default YUM/APT repositories
    • local - Installation path for packages built with build-*.sh
    • ccache - Default ccache location for docker builds
  • cmake - Additional CMake modules
  • debian - Configurations for dpkg-buildpackage
  • doc - Documentation source
  • scripts - Build scripts
    • docker - Cached Dockerfiles
      • builder - Bare images with only LStore dependencies and build tools installed
      • buildslave - Larger image with a number of developer tools
  • src - Source repositories
  • test - Test, benchmark, fuzz harness and cases
  • vendor - External dependencies

Dependencies

You will need to bring your own copies of:

  • czmq-devel
  • fuse-devel
  • leveldb-devel
  • openssl-devel
  • zlib-devel
  • zmq-devel

In addition, LStore has build-time dependencies on

  • C, C++ compiler
  • cmake

For centos, at least, these dependencies can be installed with:

yum groupinstall "Development Tools"
yum install cmake openssl-devel czmq-devel zmq-devel zlib-devel fuse-devel leveldb-devel

If the local CMake installation is too old, we install a local copy into build/

Building

LStore uses CMake as its meta build system. To initialize the build system, execute:

cd build
cmake ..

Once the Makefile is initialized, commonly used targets include:

  • make externals - build any neccessary external packages
  • make all - build LStore libraries and binaries
  • make docs - build LStore documentation

Packaging LStore

LStore uses a docker-based system for packaging LStore for various linux distributions. In general, the packaging scripts all accept a list of distributions on the command line. By default, each distribution will be attempted. These base images containing external dependencies and build tools can be bootstrapped with:

./scripts/build-docker-base.sh [distribution] [distribution] ...

For each supported distribution, a docker image named lstore/builder:DIST will pe produced and tagged. For instance, a base Centos 7 image will be named lstore/builder:centos-7. These images can be updated by executing build-docker-base.sh again.

Once the base images are installed, the current source tree can be packaged with:

./scripts/package.sh [distribution] [distribution] ...

Once package.sh completes, the output binaries for each distribution will be stored in package/<distribution>/<package>/<revision>. The revisions are auto-generated by a heuristic that considers the number of git commits between the working copy and the most recent release tag.