Nanolat Database supporting 1M transactions per second.
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
external-libs
src
.gitignore
LICENSE
Makefile
README.md
prepare-build.sh

README.md

Nanolat Database

What is it?

  • An in-memory key/value store supporting put/get/del, and range search on cursors.
  • Supports searching keys by rank. (Ex> Get 100 keys from the 1000th key)
  • Supports transactions and master-slave replication.
  • Adds transactional log replication feature on top of LevelDB and TokyoCabinet.
  • Supports Table Plugins to implement your own data structure for efficient data processing for your specific domain.

Target systems

  • Leaderboard servers sorting users by scores, getting the rank of a user.
  • Financial systems such as HFT(High Frequency Trading), FEP(Front End Processor), etc.
  • Monitoring systems that require processing huge amount of data in a second.

Why use it?

  • Low latency data processing, 1 transaction takes < 5 us.
  • Supports 1.5M transactions / second .
  • Supports built in Table Plug-ins using LevelDB and TokyoCabinet providing replication feature on top of them.
  • (Under Development in another project) Supports basic SQL that is directly converted to Nanolat Database API at compile time to achieve low latency data processing.

How it works?

  • Uses 40G Infiniband and 10G Ethernet instead of TCP/IP to replicate data from a server to another server within 2.5 us.
  • Instead of flushing transactional log on local disk, transfer it to slave machine.

How to build?

  • In MacOS, you need to install gcc 4.7 using MacPorts.

    • Download MacPorts from http://www.macports.org/install.php

    • Install gcc 4.7 using MacPorts by running following two commands.

      sudo port install gcc47

      sudo port select --set gcc mp-gcc47

    • gcc 4.6 in MacOS is not supported yet.

  • Simply run 'make prepare-build' to prepare compiler and build required libraries. Currently the script was tested on CentOS 6.3, Ubuntu 12.10, and MacOS 10.7

    • Install required libraries.
    • Check gcc version and install gcc 4.6.3 if gcc version is neither 4.6 nor 4.7.
    • Build external libraries in external-libs.
    • Setup environment variable LD_LIBRARY_PATH(Or DYLD_LIBRARY_PATH in OSX).
  • And then, run 'make'. It will build following binaries.

    • Build src/nldb/libnldb.so, src/tests/tests, src/gtest/gtest.
  • After the script finishes, you have following binaries.

    • src/nldb/libnldb.so : The shared library file of Nanolat Database
    • src/tests/tests : The testing program to run master/slave process of Nanolat Database.
    • src/gtest/gtest : The test program implements unit tests with Google Test Framework.
  • Optionally, you can use eclipse for your development environment, but you need to run build.sh at least once to prepare prerequisites and dependent libraries.

    • run eclipse, switch your workspace to nldb/src .
    • Import all projects under nldb/src .
    • clean nldb, gtest, tests projects.
    • build nldb, gtest, tests projects.

How to run a test for the master-slave replication example?

Before running a test, build it by running build.sh . Before running a test, set LD_LIBRARY_PATH by executing env.sh command after going to the src directory.

cd src
source env.sh

Create a directory for a slave database, and start a slave process of the tests program.

cd src
source env.sh
mkdir slave
cd slave
cp ../tests .
./tests nldb-slave

Start a master process.

cd src
source env.sh
./tests nldb-master

How to compile and link my program?

Add following directories as additional include paths.

src/include
src/txbase

Specify std=c++0x to compile your program using c++0x.

For example, in case you use g++, use following command line to compile your program where ${NLDB_HOME} has this README.md file.

g++ -I${NLDB_HOME}/src/include -I${NLDB_HOME}/src/txbase -c -std=c++0x -o your_program.o your_program.cpp

Add following directories as additional library paths.

external-libs/lib
src/nldb

To link your program, add following libraries.

nldb
leveldb
tokyocabinet
xs

For example, in case you use g++, use following command line to link your program where ${NLDB_HOME} has this README.md file.

g++ -L{NLDB_HOME}/external-libs/lib -L{NLDB_HOME}/nldb -o your_program_binary  your_program.o -lnldb -lleveldb -ltokyocabinet -lxs

Apache v2 License

http://www.apache.org/licenses/LICENSE-2.0.html