Building Fluentd

Building Fluentd

The instructions provided below specify the steps to build Fluentd 1.16.5 on Linux on IBM Z for the following distributions:

• RHEL (7.8, 7.9, 8.6, 8.8, 8.9, 9.0, 9.2, 9.3)
• SLES (12 SP5, 15 SP5)
• Ubuntu (20.04, 22.04, 23.10)

General Notes:

• When following the steps below please use a standard permission user unless otherwise specified.

• A directory /<source_root>/ will be referred to in these instructions, this is a temporary writable directory anywhere you'd like to place it.

Step 1: Build and Install Fluentd

1.1) Build using script

If you want to build fluentd using manual steps, go to STEP 1.2.

Use the following commands to build fluentd using the build script. Please make sure you have wget installed.

wget -q https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Fluentd/1.16.5/build_fluentd.sh

# Build fluentd
bash build_fluentd.sh

If the build completes successfully, go to STEP 2. In case of error, check logs for more details or go to STEP 1.2 to follow manual build steps.

1.2) Install dependencies

export SOURCE_ROOT=/<source_root>/

• Ubuntu (20.04, 22.04, 23.10)

  sudo apt update && sudo apt install -y zlib1g-dev wget

• RHEL (7.x, 8.x)

  sudo yum install wget 

• RHEL (9.0, 9.2, 9.3)

  sudo yum install -y rpmdevtools zlib-devel zlib wget

• SLES (12 SP5, 15 SP5)

  sudo zypper install -y gzip awk zlib-devel wget

• Build Ruby

  wget -q https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Ruby/3.2.2/build_ruby.sh
  bash build_ruby.sh -y

1.3) Setup the GEM_HOME and PATH environment for a standard user (For RHEL)

export GEM_HOME=${HOME}/.gem/ruby
export PATH=${HOME}/.gem/ruby/bin:$PATH

1.4) Build and install the latest stable fluentd gem and dependencies

Fluentd is a gem based product, so the rubygem install process will automatically build the necessary native parts

• RHEL

  gem install fluentd -v 1.16.5

• Ubuntu & SLES

  sudo gem install fluentd -v 1.16.5

Once complete it should report that a number of gems including fluentd are installed. Verify the installed version with gem list fluentd

Note: In case of error gem command not found on SLES, run above command as sudo env PATH=$PATH gem install fluentd -v 1.16.5.

Step 2: Testing (Optional)

2.1) Quick test of the fluentd installation

This test is to setup/install a config directory, then start a fluent process which is put into the background. Finally a message is piped to fluent-cat

fluentd -s conf
fluentd -c conf/fluent.conf &
echo '{"json":"message"}' | fluent-cat debug.test

This should display similar to the following:

2023-11-22 12:31:51.243234629 +0000 debug.test: {"json":"message"}

If the fluent process is not running the response will be similar to the following:

connect failed: Connection refused - connect(2) for "127.0.0.1" port 24224

Note: Typing fluentd -help gives an outline of fluentd commands available. If it throws error fluentd command not found, set fluentd binary path to PATH environment variable.

References:

• https://github.com/fluent/fluentd