cppkafka: high level C++ wrapper for rdkafka

cppkafka allows C++ applications to consume and produce messages using the Apache Kafka protocol. The library is built on top of librdkafka, and provides a high level API that uses modern C++ features to make it easier to write code while keeping the wrapper's performance overhead to a minimum.

Features

• cppkafka is a high level C++ wrapper for rdkafka, aiming to allow using rdkafka in a simple, less error prone way.

• cppkafka provides an API to produce messages as well as consuming messages, but the latter is only supported via the high level consumer API. cppkafka requires rdkakfa >= 0.9.4 in order to use it. Other wrapped functionalities are also provided, like fetching metadata, offsets, etc.

• cppkafka tries to add minimal overhead over librdkafka. A very thin wrapper for librdkafka messages is used for consumption so there's virtually no overhead at all.

It's simple!

cppkafka's API is simple to use. For example, this code creates a producer that writes a message into some partition:

#include <cppkafka/producer.h>

using namespace std;
using namespace cppkafka;

int main() {
    // Create the config
    Configuration config = {
        { "metadata.broker.list", "127.0.0.1:9092" }
    };

    // Create the producer
    Producer producer(config);

    // Produce a message!
    string message = "hey there!";
    producer.produce(MessageBuilder("my_topic").partition(0).payload(message));
}

Compiling

In order to compile cppkafka you need:

• librdkafka >= 0.9.4
• CMake
• A compiler with good C++11 support (e.g. gcc >= 4.8). This was tested successfully on g++ 4.8.3.
• The boost library. cppkafka only requires boost.optional, which is a header only library, so this doesn't add any additional runtime dependencies.

Now, in order to build, just run:

mkdir build
cd build
cmake ..
make

CMake options

If you have installed librdkafka on a non standard directory, you can use the RDKAFKA_ROOT_DIR cmake parameter when configuring the project:

cmake .. -DRDKAFKA_ROOT_DIR=/some/other/dir

Note that finding librdkafka will succeed iff there's an include and lib directories inside the specified path, including both the rdkafka.h header and the librdkafka library file.

---

By default, a shared library will be built. If you want to perform a static build, use the CPPKAFKA_BUILD_SHARED parameter:

cmake .. -DCPPKAFKA_BUILD_SHARED=0

Using

If you want to use cppkafka, you'll need to link your application with:

• cppkafka
• rdkafka

Documentation

You can generate the documentation by running make docs inside the build directory. This requires Doxygen to be installed. The documentation will be written in html format at <build-dir>/docs/html/.

Make sure to check the wiki which includes some documentation about the project and some of its features.