Copyright (c) 2012-2013, Magnus Edenhill.
https://github.com/edenhill/librdkafka
librdkafka is a C library implementation of the Apache Kafka protocol, containing both Producer and Consumer support. It was designed with message delivery reliability and high performance in mind, current figures exceed 800000 msgs/second for the producer and 3 million msgs/second for the consumer.
librdkafka is licensed under the 2-clause BSD license.
For an introduction to the performance and usage of librdkafka, see INTRODUCTION.md
NOTE: The master branch is actively developed, use latest release for production use.
Apache Kafka 0.8 support:
- Branch: master
- Producer: supported
- Consumer: supported
- Compression: snappy and gzip
- Debian package: librdkafka1 and librdkafka-dev in Debian and Ubuntu
- ZooKeeper: not supported
- C API: Stable, ABI safe, not backwards compatible with 0.7
- C++ API: Testing
- Tests: Regression tests in
tests/directory. - Statistics: JSON formatted, see
rd_kafka_conf_set_stats_cbinrdkafka.h. - Status: Stable
Apache Kafka 0.7 support:
- Branch: 0.7
- Producer: supported
- Consumer: supported
- Compression: not supported
- ZooKeeper: not supported
- C API: backwards compatible with 0.6
- Status: Stable
Apache Kafka 0.6 support:
- Branch: 0.6
- Producer: supported
- Consumer: supported
- Compression: not supported
- ZooKeeper: not supported
- Status: Testing
#Users of librdkafka#
- Wikimedia's varnishkafka - Varnish cache web log producer
- kafkacat - Apache Kafka swiss army knife
- redBorder
- Headweb
- Produban's log2kafka - Web log producer
- phpkafka - PHP
- node-kafka - Node.js
- node-kafkacat - Node.js
- haskakafka - Haskell
- haskell-kafka - Haskell
- Hermann - Ruby
- large unnamed financial institution
- Let me know if you are using librdkafka
The GNU toolchain
GNU make
pthreads
zlib
./configure
make
sudo make install
See examples/rdkafka_example.c for an example producer and consumer.
Link your program with -lrdkafka -lz -lpthread -lrt.
The C API is documented in src/rdkafka.h
The C++ API is documented in src-cpp/rdkafkacpp.h
Configuration properties are documented in CONFIGURATION.md
For a librdkafka introduction, see INTRODUCTION.md
See the examples/sub-directory.
See the tests/sub-directory.
File bug reports, feature requests and questions using GitHub Issues
Questions and discussions are also welcome on irc.freenode.org, #apache-kafka, nickname Snaps.
Commercial support is available from Edenhill services