This repository contains the scripts used for the paper "Don't Hold My Data Hostage - A Case For Client Protocol Redesign", as well as a link to the VM that can be used to repeat these experiments.
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
README.md
benchmark.py
graph.R

README.md

This set of benchmarks measures the performance of database (and other systems') client protocols when transferring a large amount of data. The following systems are measured:

  • PostgreSQL
  • MariaDB (MySQL)
  • DB2
  • Oracle
  • MonetDB
  • MongoDB
  • Hive

All the benchmarks are performed using the ODBC connector of the database (except for Hive, which uses the JDBC connector). However, there is also testing code for benchmarking the native client connector and the JDBC client connector of each of the databases.

In addition to the current systems, we measure the implementation of a new protocol in PostgreSQL and MonetDB. The implementation of these can be found here:

Paper

Mark Raasveldt and Hannes Mühleisen: Don't Hold My Data Hostage - A Case For Client Protocol Redesign, 43rd International Conference on Very Large Data Bases (VLDB2017) PDF

VM Image Download

All the benchmarks are performed on a VM running Ubuntu 16.04. The image of the VM can be downloaded here. The login credentials to the VM are username: user, password: user. The experiments can be run by starting the VM and running the benchmark script using the command python benchmark.py. The results of the experiments will be put in the /home/user/results folder in CSV format. The graphs/tables can be generated using the command R -f graph.R.

The different database systems can be started as follows:

PostgreSQL
sudo service postgresql start
MySQL
sudo service mysql start
DB2
db2start
Oracle
docker run -d -p 49160:22 -p 49161:1521 -v /home/user:/opt/user wnameless/oracle-xe-11g
Hive
$HIVE_HOME/bin/hive --service hiveserver2
MonetDB
/home/user/monetdb-install/bin/mserver5 --set gdk_nr_threads=1 --set mapi_port=50001 --dbpath=/home/user/monetdb/database --set varchar_maximum_fixed=3 --set optimizer_pipeline=sequential
PostgreSQL++
export PGDATA=/home/user/Sources/postgres/pgdata
/home/user/Sources/postgres/build/bin/postgres -p 5678
PostgreSQL++C
export PGDATA=/home/user/Sources/postgres/pgdata_compress
POSTGRES_COMPRESSION=true /home/user/Sources/postgres/build/bin/postgres -p 5700

Additional Notes

For the MongoDB ODBC driver this proprietary driver is used. This driver requires a valid license file called SimbaMongoDBODBCDriver.lic to be in the users' home directory. This license file can be obtained by requesting a free trial of the software, after which they will email you a license file. In case this driver is no longer available, we have archived it as well.