C C++ Shell Python CMake Objective-C Other
Permalink
Failed to load latest commit information.
.github Add OS to template May 15, 2016
examples cmake: add simple producer Mar 20, 2017
mklove mklove: fix static linking on osx Feb 20, 2017
packaging rpm: add lz4-devel to BuildRequires Feb 16, 2017
src-cpp C++: fix conf->get() allocation (closes #1118) Mar 16, 2017
src Improve broker thread responsiveness Mar 22, 2017
tests test 0061: make consumer lag tests more robust Mar 21, 2017
win32 Use app_offset to calculate consumer_lag (closes #1112) Mar 16, 2017
.dir-locals.el emacs default compile command Jan 7, 2017
.doozer.json doozer: separate tests into testcmd Feb 19, 2017
.gitignore Ignore some test artifacts Jan 17, 2017
.travis.yml travis: s3 deploy from all branches (and for PRs) for now Feb 20, 2017
CMakeLists.txt cmake: Add install target (#1027) Feb 10, 2017
CONFIGURATION.md Improve bootstrap.servers docs Mar 16, 2017
CONTRIBUTING.md CONTRIBUTING: 8 space indent, not 4 Nov 15, 2016
Doxyfile Added @locks doxygen tag Jan 7, 2017
INTRODUCTION.md More metadata refactoring, caching, fixes, overhaul. Jan 7, 2017
LICENSE Extracted rdkafka from librd to its own library. Sep 19, 2012
LICENSE.pycrc Extracted rdkafka from librd to its own library. Sep 19, 2012
LICENSE.queue Native Win32 (VS) support (take one) Mar 24, 2015
LICENSE.regexp regexp import from ccxvii/minilibs sha 875c33568b5a4aa4fb3dd0c52ea98f… Sep 28, 2016
LICENSE.snappy snappy-c upstream update: Clean import of andikleen/snappy-c@8015f2d Nov 5, 2015
LICENSE.tinycthread clean tinycthread import tinycthread.c at ce82058b05180ce, tinycthrea… Mar 23, 2015
LICENSE.wingetopt clean wingetopt import of wingetopt.c and wingetopt.h at version 281d… Mar 23, 2015
LICENSE.xxhash Added support for LZ4 compression (KIP-57) Jun 2, 2016
LICENSES.txt Add LICENCES.txt containing all licences Nov 21, 2016
Makefile Add LICENCES.txt containing all licences Nov 21, 2016
README.md Adding Swift to Language Bindings Mar 8, 2017
README.win32 Win32: add lz4 support (using NuGet lz4 package) Nov 21, 2016
config.h.in Add CMake (#1014, @ruslo) Jan 27, 2017
configure mklove: fix ./configure --reconfigure Nov 24, 2016
configure.librdkafka configure: auto add brew openssl pkg-config path Mar 22, 2017
dev-conf.sh dev-conf: use AddressSanitizer Jan 7, 2017
lds-gen.py Dont include static functions in linker script (issue #662) May 18, 2016
mainpage.doxy Doxygenification: C API, C++ API stub, INTRODUCTION, CONFIGURATION an… Dec 8, 2015

README.md

librdkafka - the Apache Kafka C/C++ client library

Copyright (c) 2012-2016, Magnus Edenhill.

https://github.com/edenhill/librdkafka

Gitter chat Build status

librdkafka is a C library implementation of the Apache Kafka protocol, containing both Producer and Consumer support. It was designed with message delivery reliability and high performance in mind, current figures exceed 1 million msgs/second for the producer and 3 million msgs/second for the consumer.

librdkafka is licensed under the 2-clause BSD license.

For an introduction to the performance and usage of librdkafka, see INTRODUCTION.md

See the wiki for a FAQ.

NOTE: The master branch is actively developed, use latest release for production use.

#Overview#

  • High-level producer
  • High-level balanced KafkaConsumer (requires broker >= 0.9)
  • Simple (legacy) consumer
  • Compression: snappy, gzip, lz4
  • SSL support
  • SASL (GSSAPI/Kerberos, PLAIN) support
  • Broker version support: >=0.8 (see Broker version compatibility)
  • Stable C & C++ APIs (ABI safety guaranteed for C)
  • Statistics metrics
  • Debian package: librdkafka1 and librdkafka-dev in Debian and Ubuntu
  • RPM package: librdkafka and librdkafka-devel
  • Portable: runs on Linux, OSX, Win32, Solaris, FreeBSD, ...

#Language bindings#

#Users of librdkafka#

Usage

Requirements

The GNU toolchain
GNU make
pthreads
zlib (optional, for gzip compression support)
libssl-dev (optional, for SSL support)
libsasl2-dev (optional, for SASL support)
liblz4-dev (optional, for LZ4 compression support)

Instructions

Building

  ./configure
  make
  sudo make install

NOTE: See README.win32 for instructions how to build on Windows with Microsoft Visual Studio.

Usage in code

See examples/rdkafka_example.c for an example producer and consumer.

Link your program with -lrdkafka -lz -lpthread -lrt.

Documentation

The public APIs are documented in their respective header files:

To generate Doxygen documents for the API, type:

make docs

Configuration properties are documented in CONFIGURATION.md

For a librdkafka introduction, see INTRODUCTION.md

Examples

See the examples/sub-directory.

Tests

See the tests/sub-directory.

Support

File bug reports, feature requests and questions using GitHub Issues

Questions and discussions are also welcome on irc.freenode.org, #apache-kafka, nickname Snaps.

Commercial support

Commercial support is available from Edenhill services