Building librdkafka

Building librdkafka

Below versions of librdkafka are available in respective distributions at the time of creation of these build instructions:

• RHEL (7.8, 7.9) have 0.11.4-1.el7
• RHEL (8.6, 8.8, 8.9) have 0.11.4-3.el8
• RHEL (9.0, 9.2, 9.3) have 1.6.1-102.el9
• Ubuntu 22.04 has 1.8.0-1build1
• Ubuntu 20.04 has 1.2.1-1ubuntu1
• Ubuntu 23.10 has 2.2.0-1
• SLES 12 SP5 has 0.11.6-1.3.1
• SLES 15 SP5 has 0.11.6-1.8.1

The instructions provided below specify the steps to build librdkafka version 2.3.0 on Linux on IBM Z for following distributions:

• RHEL (7.8, 7.9, 8.6, 8.8, 8.9, 9.0, 9.2, 9.3)
• SLES (12 SP5, 15 SP5)
• Ubuntu (20.04, 22.04, 23.10)

General Notes:

• When following the steps below please use a standard permission user unless otherwise specified.
• A directory /<source_root>/ will be referred to in these instructions, this is a temporary writable directory anywhere you'd like to place it.

Step 1: Build and Install librdkafka

1.1) Build using script

If you want to build librdkafka using manual steps, go to STEP 1.2.

Use the following commands to build librdkafka using the build script. Please make sure you have wget installed.

wget -q https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/librdkafka/2.3.0/build_librdkafka.sh

# Build librdkafka
bash build_librdkafka.sh   [Provide -t option for executing build with tests]

If the build completes successfully, go to STEP 2. In case of error, check logs for more details or go to STEP 1.2 to follow manual build steps.

1.2) Install the build dependencies

export SOURCE_ROOT=/<source_root>/

• RHEL (7.8, 7.9, 8.6, 8.8, 8.9, 9.0, 9.2, 9.3)

  sudo yum install -y git openssl-devel cyrus-sasl-devel python3 gcc gcc-c++ zlib-devel binutils make
  

• SLES (12 SP5)

  sudo zypper install -y binutils gcc make libz1 zlib-devel libsasl2-2 libzstd-devel git gcc-c++ openssl-devel which

• SLES 15 SP5

  sudo zypper install -y binutils gcc make libz1 zlib-devel git gcc-c++ openssl-devel

• Ubuntu (20.04, 22.04, 23.10)

  sudo apt-get update
  sudo apt-get install -y git build-essential make zlib1g-dev libpthread-stubs0-dev libssl-dev libsasl2-dev libzstd-dev libcurl4-openssl-dev

1.3) Download, configure and install librdkafka

• Download source

  cd $SOURCE_ROOT
  git clone https://github.com/confluentinc/librdkafka.git
  cd librdkafka/
  git checkout v2.3.0

• Configure

  ./configure --install-deps

• Build and install

  make
  sudo make install

Step 2: Testing (Optional)

2.1) Run the unit and integration tests

cd $SOURCE_ROOT/librdkafka
make -C tests run_local_quick

Step 3: Verify (Optional)

3.1) set LD_LIBRARY_PATH

export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH

3.2) Create verify_librdkafka.c

#include<stdio.h>
#include<librdkafka/rdkafka.h>

   int main(){
   printf("librdkafka version: %s\n", rd_kafka_version_str());
   return 0;
   }

3.3) Compile and run

gcc -o verify_librdkafka verify_librdkafka.c -lrdkafka
./verify_librdkafka

References:

• https://github.com/confluentinc/librdkafka#documentation