Nanolat Database supporting 1M transactions per second.
C++ C Shell Objective-C
Switch branches/tags
Nothing to show
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
external-libs
src
.gitignore
LICENSE
Makefile
README.md
prepare-build.sh

README.md

Nanolat Database

What is it?

  • An in-memory key/value store supporting put/get/del, and range search on cursors.
  • Supports searching keys by rank. (Ex> Get 100 keys from the 1000th key)
  • Supports transactions and master-slave replication.
  • Adds transactional log replication feature on top of LevelDB and TokyoCabinet.
  • Supports Table Plugins to implement your own data structure for efficient data processing for your specific domain.

Target systems

  • Leaderboard servers sorting users by scores, getting the rank of a user.
  • Financial systems such as HFT(High Frequency Trading), FEP(Front End Processor), etc.
  • Monitoring systems that require processing huge amount of data in a second.

Why use it?

  • Low latency data processing, 1 transaction takes < 5 us.
  • Supports 1.5M transactions / second .
  • Supports built in Table Plug-ins using LevelDB and TokyoCabinet providing replication feature on top of them.
  • (Under Development in another project) Supports basic SQL that is directly converted to Nanolat Database API at compile time to achieve low latency data processing.

How it works?

  • Uses 40G Infiniband and 10G Ethernet instead of TCP/IP to replicate data from a server to another server within 2.5 us.
  • Instead of flushing transactional log on local disk, transfer it to slave machine.

How to build?

  • In MacOS, you need to install gcc 4.7 using MacPorts.

    • Download MacPorts from http://www.macports.org/install.php

    • Install gcc 4.7 using MacPorts by running following two commands.

      sudo port install gcc47

      sudo port select --set gcc mp-gcc47

    • gcc 4.6 in MacOS is not supported yet.

  • Simply run 'make prepare-build' to prepare compiler and build required libraries. Currently the script was tested on CentOS 6.3, Ubuntu 12.10, and MacOS 10.7

    • Install required libraries.
    • Check gcc version and install gcc 4.6.3 if gcc version is neither 4.6 nor 4.7.
    • Build external libraries in external-libs.
    • Setup environment variable LD_LIBRARY_PATH(Or DYLD_LIBRARY_PATH in OSX).
  • And then, run 'make'. It will build following binaries.

    • Build src/nldb/libnldb.so, src/tests/tests, src/gtest/gtest.
  • After the script finishes, you have following binaries.

    • src/nldb/libnldb.so : The shared library file of Nanolat Database
    • src/tests/tests : The testing program to run master/slave process of Nanolat Database.
    • src/gtest/gtest : The test program implements unit tests with Google Test Framework.
  • Optionally, you can use eclipse for your development environment, but you need to run build.sh at least once to prepare prerequisites and dependent libraries.

    • run eclipse, switch your workspace to nldb/src .
    • Import all projects under nldb/src .
    • clean nldb, gtest, tests projects.
    • build nldb, gtest, tests projects.

How to run a test for the master-slave replication example?

Before running a test, build it by running build.sh . Before running a test, set LD_LIBRARY_PATH by executing env.sh command after going to the src directory.

cd src
source env.sh

Create a directory for a slave database, and start a slave process of the tests program.

cd src
source env.sh
mkdir slave
cd slave
cp ../tests .
./tests nldb-slave

Start a master process.

cd src
source env.sh
./tests nldb-master

How to compile and link my program?

Add following directories as additional include paths.

src/include
src/txbase

Specify std=c++0x to compile your program using c++0x.

For example, in case you use g++, use following command line to compile your program where ${NLDB_HOME} has this README.md file.

g++ -I${NLDB_HOME}/src/include -I${NLDB_HOME}/src/txbase -c -std=c++0x -o your_program.o your_program.cpp

Add following directories as additional library paths.

external-libs/lib
src/nldb

To link your program, add following libraries.

nldb
leveldb
tokyocabinet
xs

For example, in case you use g++, use following command line to link your program where ${NLDB_HOME} has this README.md file.

g++ -L{NLDB_HOME}/external-libs/lib -L{NLDB_HOME}/nldb -o your_program_binary  your_program.o -lnldb -lleveldb -ltokyocabinet -lxs

Apache v2 License

http://www.apache.org/licenses/LICENSE-2.0.html