Skip to content
kdb+ to Apache Kafka adapter, for pub/sub
Branch: master
Clone or download
sv Merge pull request #13 from sv/master
managing offsets api
Latest commit 9032b8e Nov 7, 2018
Type Name Latest commit message Commit time
Failed to load latest commit information.
Makefile Merge pull request #13 from sv/master Nov 7, 2018
socketpair.c initial release May 31, 2017
test_offsetc.q add example of working with offsets Sep 12, 2018
test_offsetp.q add example of working with offsets Sep 12, 2018
test_producer.q initial release May 31, 2017

kfk – Kafka for kdb+

kfk is a thin wrapper for kdb+ around librdkadka C API for Kafka. It is part of the Fusion for kdb+ interface collection.

Please report issues in this repository.

See for full documentation.

Build and install

Step 1

Build and install the latest version of librdkafka. The minimum required version is v0.11.0.

Step 1

Build and install latest version of librdkafka. Minimum required version is v0.11.0.


Follow requirements for librdkafka compilation noted on librdkafka page

To build 32-bit versions on 64-bit OS you need to have 32-bit versions of libraries and a toolchain.

sudo yum install glibc-devel.i686 libgcc.i686 libstdc++.i686 zlib-devel.i686
# Ubuntu
sudo apt-get install gcc-multilib


Package installation


brew install librdkafka


sudo apt-get install librdkafka-dev


sudo yum install librdkafka-devel

Building from source

macOS and Linux

git clone
cd librdkafka
make clean  # to make sure nothing left from previous build or if upgrading/rebuilding
# If using OpenSSL, remove --disable-ssl from configure command below
# On macOS with OpenSSL you might need to set `export OPENSSL_ROOT_DIR=/usr/local/Cellar/openssl/1.0.2k` before proceeding

# 32 bit
./configure --prefix=$HOME --disable-sasl --disable-lz4 --disable-ssl --mbits=32 
# 64 bits
./configure --prefix=$HOME --disable-sasl --disable-lz4 --disable-ssl --mbits=64

make install

Step 2

Compile and install a shared object (it will be installed to $QHOME/). Make sure you have QHOME environment set.

// in kfk source folder
make install

Note: If compiling dynamically linked make sure you have in your LD_LIBRARY_PATH.




To have launchd start kafka now and restart at login: brew services start kafka Or, if you don't want/need a background service you can just run: zookeeper-server-start /usr/local/etc/kafka/ & kafka-server-start /usr/local/etc/kafka/

You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.