gdbb

(Graph DB benchmarks)

Benchmark link-prediction algorithms overs different DBMS

Algorithm information can be found in The Link Prediction Problem for Social Networks

Graphs are extracted from the DBLP Computer Science Bibliography XML, and from Stanford Large Network Dataset Collection

Results can be found here

Prerequisites

axel (to download DBLP, can be replaced with wget)
bash
python 2.7
python-lxml
python-mysqldb
python-redis
mysql 5.5
redis-server
Neo4J Community 1.9.M03
Oracle JDK 6
py2neo

Quick Start

Fetch code:

git clone https://github.com/natict/gdbb.git
cd gdbb

Generate graphs:

./mkgraphs.sh

Create MySQL tables and procedures (you'll need mysql root), e.g. for all_core3:

./init-mysql.sh all_core3

Execute MySQL benchmark:

./benchmark_mysql.py all_core3

Execute Redis benchmark (i.e. for all_core3):

./benchmark_redis.py -d all_core3

Create Neo4J graphs (you'll need to be a sudo-er) (i.e. for all_core3):

./init-neo4j.sh all_core3

Execute Neo4J benchmark:

./benchmark_neo4j.py all_core3

Execute all benchmarks (assuming sudo permissions):

./benchmark-all.sh [MySQL_root_password]

Notes

Graph generation is done in-memory, make sure you have enough
Some MySQL configuration changes are recommended (significantly speedup load time, allow recursive procedures):

innodb_additional_mem_pool_size = 512M
innodb_buffer_pool_size = 512M
innodb_log_file_size = 256M
innodb_log_buffer_size = 256M
max_sp_recursion_depth=10

Using the latest versions of redis-server and redis-py significantly improves performance
Currently the Lua scripts are redis-calls optimized and NOT memory optimized
Some Redis settings are required:

lua-time-limit 600000	# 10 Minutes

Let the Neo4J JVM to allocate more heap (conf/neo4j-wrapper.conf)

# Initial Java Heap Size (in MB)
wrapper.java.initmemory=1024

# Maximum Java Heap Size (in MB)
wrapper.java.maxmemory=16384

natict/gdbb

ErrorLooks like something went wrong!

About

Resources

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Languages

Error
Looks like something went wrong!

Packages