A C++ client library for Apache Kafka v0.8+. Also includes C API.
C++ Shell
Permalink
Failed to load latest commit information.
app updates for 0.5.0 release Dec 18, 2013
config travis CI config Sep 9, 2013
install updates for 0.5.0 release Dec 18, 2013
lib
m4 updated tests to improve gcov results Oct 11, 2013
samples added error test to SimpleProducer for connection errors Dec 19, 2013
test updates for 0.5.0 release Dec 18, 2013
tools prep for 0.3.0 release Sep 30, 2013
.gitignore added MetdataResponse class, tests Aug 13, 2013
.travis.yml updated travis CI config, coveralls config Oct 15, 2013
AUTHORS
CHANGELOG updates for 0.5.0 release Dec 18, 2013
COPYING updated license and headers for the Apache 2.0 license Sep 3, 2013
LICENSE updated license and headers for the Apache 2.0 license Sep 3, 2013
Makefile.am updates for 0.5.0 release Dec 18, 2013
README.md Added SimpleProducer sample Dec 18, 2013
autogen.sh added initial autoconf stuff. still a little buggy... Aug 19, 2013
configure.ac updates for 0.5.0 release Dec 18, 2013
pom.xml
valgrind.sh
valgrind.supp memory management fixes to clean up valgrind Aug 20, 2013

README.md

libkafka Build Status Coverage Status

A C++ client library for Apache Kafka v0.8+. Also includes C API. With a full unit test suite.

Supports the following Kafka API request/response calls:

  • Metadata
  • Produce
  • Fetch
  • Offset

Includes support for MessageSet compression using GZIP or Snappy compression.

Not compatible with Kafka versions prior to 0.8 due to wire protocol changes.

Version

Current release version: 0.5.0 (see RELEASES)

Dependencies

  • zlib for CRC calculation, GZIP compression
  • Snappy for Snappy compression
  • Google Test for unit tests
  • (optional) lcov/gcov for unit test coverage reporting
  • (optional) doxygen for C/C++ API documentation

Installation

Supports both autoconf and maven builds:

Autoconf (g++)

  • ./autogen.sh
  • ./configure
  • make (builds shared library)
  • make check (runs protocol unit tests, expects a kafka broker at host:port and with topics as specified in TestConfig.cc)
  • make rpm (packages an RPM, tested on CentOs 6.x, genearted into ./rpm/RPMS/$(arch))
  • make distcheck (produces release packages, tar.gz and tar.bz2)
  • make maintainer-clean (clean all autoconf generated files)
  • make rpm-clean (clean all artifacts of rpm, dist, and autotools)

Unit test coverage reports (gcov/lcov)

  • ./autogen.sh && ./configure --enable-gcov && make && make check && make coverage-html
  • HTML reports produced in ./coveragereport

Maven (clang)

  • mvn clean install (builds shared library, command line apps)
  • mvn test (runs protocol unit tests)

Debugging

  • Make with -DDEBUG or uncomment -DDEBUG in the top level pom.xml, and rebuild for full protocol debugging output.

Valgrind

  • Valgrind currently provides a clean memory leak report for the unit test suite. Future commits will maintain this.
  • ./valgrind.sh runs the valgrind tests with suppressions for Mac OS X.

Examples

High-level examples:

  • Take a look at the samples directory.

Low-level examples:

  • Take a look at ClientTest.cc in test/src. There are examples for Metadata, Produce, Fetch, and Offsets, with all of the compression combinations.
  • Take a look at the unit tests in test/src to see examples for any individual class.

TODO

  • Advanced client with error handling, metadata/broker/leader updates, and offset management
  • load testing for Produce and Fetch

Known issues

  • Nested MessageSet blocks are not yet handled. This is a wonky part of the protocol.
  • The extern C wrrapper for C++ calls is not yet implemented.
  • The .deb packaging remains broken. RPM packaging is fully operational.

Fixed Issues

  • See CHANGELOG

License

Licensed under the Apache Software License 2.0. See LICENSE file.