RocksDB Python

Cross-platform Python bindings for RocksDB. Other RocksDB Python bindings libraries usually don't provide Windows builds. This repo aims to target all OSes, including Windows.

This project helps me to learn using C++ libraries in Python with pybind11.

Installation

From wheels

TBD

From source

Build RocksDB

You need to build RocksDB (and optionally 3rd-party compression libraries) from source. Refer to the official instructions for more information. Additionally, you can also refer to Building on Windows if you are building on Windows.

# on Linux and macOS
git clone --recursive https://github.com/gau-nernst/rocksdb-python
cd rocksdb-python/rocksdb   # rocksdb v7.3.1
make static_lib -j4
cp librocksdb.a ..

I have GitHub Actions Workflows that build RocksDB binaries and all supported 3rd-party compression libraries. Refer to .github/workflows/build_rocksdb_{linux/mac/win}.yaml for my specific build commands. Thus you can also download my pre-built binaries from GitHub Actions (look for the artifacts under Build RocksDB for Linux/macOS/Windows). macOS builds are universal2 (support both Apple Intel x86_64 and Apple Silicon ARM64).

OS	Arch	Snappy	LZ4	Zlib	Zstd	Bzip2	XPRESS
Linux	x86_64	✅	✅	✅	✅	✅
macOS	universal2	✅	✅	✅	✅	✅
Windows	x86_64	✅	✅	✅	✅		✅ (comes with Windows)

Place the built binaries in the root directory of this repo. e.g. librocksdb.a on Linux and macOS, rocksdb.lib on Windows.

Build Python bindings (this repo)

# if you haven't cloned
git clone --recursive https://github.com/gau-nernst/rocksdb-python
cd rocksdb-python

pip install pybind11
python setup.py build_ext -i install
# on macOS, add MACOSX_DEPLOYMENT_TARGET=10.13, follows RocksDB
# MACOSX_DEPLOYMENT_TARGET=10.13 python setup.py build_ext -i install

The rocksdb submodule should be in the same commit as the commit used to build RocksDB in the previous step (or even better, same release version). This is to make sure RocksDB's header files are consistent with the RocksDB binary.

Basic usage

from rocksdb_python import Options, PyDB, ReadOptions, WriteOptions

options = Options()
options.IncreaseParallelism()
options.create_if_missing = True

db = PyDB(options, "./test.db")
db.Put(WriteOptions(), b"key", b"value")
v = db.Get(ReadOptions(), b"key")
assert v == b"value"