Analytic Distributed DBMS for the project SW-StarLab

We improve analytic DBMS, a special kind of DBMS which stores big data for analytic purposes. We are developing "ADDB (Analytic Distributed DBMS)", an in-memory, distributed version of analytic DBMS based on flash memory storage. In this project, emerging persistent storage such as NVRAM will also be concerned.

This research was supported by the MSIT(Ministry of Science and ICT), Korea, under the SW Starlab support program(IITP-2017-0-00477) supervised by the IITP(Institute for Information & communications Technology Promotion).

Prerequisites

Before compile ADDB, please execute these commands on ADDB path.

echo "export ADDB_HOME=$(pwd)" >> ~/.bash_profile
echo "export LD_LIBRARY_PATH=\$LD_LIBRARY_PATH:\$ADDB_HOME/deps/rocksdb" >> ~/.bash_profile
echo "export LD_LIBRARY_PATH=\$LD_LIBRARY_PATH:\$ADDB_HOME/deps/jemalloc/lib" >> ~/.bash_profile
source ~/.bash_profile

Run ADDB-RR

# Compile
make -j8

# Run server (with 'redis_tiering.conf')
./src/redis-server redis_tiering.conf

# Run client
./src/redis-cli

Run test_scripts

# Create fpWrite, fpScan commands
cd test_scripts/
./cmdGenerator.sh TABLE_ID=100 MODE=static PARTITION_COUNT=100 COLUMN_COUNT=4 SCAN_LOOK_UP_COLUMNS="1,2,3" INSERT_ITER_CNT=200 SCAN_ITER_CNT=30

# Run multiple fpWrite commands
cat ./test_scripts/fpWriteCmd.resp | ./src/redis-cli --pipe

# (Option) Watch server status by client
./src/redis-cli --stat

Leak check

Please check memory leaks by this method.

How to use?

Step 1: Install 'valgrind' software
http://www.valgrind.org/

Step 2: Re-compile 'redis-server' for using valgrind.

make valgrind -j8

Step 3: Run below commands when starts 'redis-server'.
Valgrind log file will be stored on '${log_file_path}'.

valgrind --leak-check=full --trace-children=yes --log-file=${log_file_path} ./src/redis-server redis_tiering.conf