Skip to content
q language bindings for Apache Kafka based on librdkafka
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Type Name Latest commit message Commit time
Failed to load latest commit information.


q language bindings for Apache Kafka based on librdkafka


  • Install librdkafka
  • Download/clone this repo and navigate to the kafkaq directory
  • On Linux, compile with the following:
gcc -O2 -fPIC -shared kafkaq.c -o -lrdkafka -lpthread

Kakfa Installation

To test with a local instance of kafka (more details here - the box should have an up to date version of java installed)

> # Download kafka package
> wget
> # Extract package and navigate to directory
> tar -xzf kafka_2.11-
> cd kafka_2.11-
> # start zookeeper server
> bin/ -daemon config/
> # start kafka server
> bin/ -daemon config/
> # create a topic called 'test'
> bin/ --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test

Usage Example

The included q file kafka.q assumes the library is available in the same folder In this example q is acting as both producer and consumer via the local kafka instance started above, a more realistic scenario q would be either producer or consumer but not both.

q)\l kafka.q
q)kupd / print default definition for incoming data - ignore key, print message as ascii
{[k;x] -1 `char$x;}
q)subscribe[`test;0] / subscribe to topic test, partition 0
q)pub:{publish[`test;0;`;`byte$x]} / define pub to publish text input to topic test on partition 0 with no key defined
q)pub"hello world"
q)hello world



  • Currently the library supports a single consumer thread subscribed to one topic at a time
You can’t perform that action at this time.