Building Apache Cassandra 4.x

Building Apache Cassandra

The instructions provided below specify the steps to build Apache Cassandra version 4.1.4 on Linux on IBM Z for the following distributions:

• RHEL (7.8, 7.9, 8.6, 8.8, 8.9, 9.0, 9.2, 9.3)
• SLES (12 SP5, 15 SP5)
• Ubuntu (20.04, 22.04, 23.10)

The binary for Apache Cassandra version 4.1.4 can be downloaded from here. Binary works for basic operations however it might not work for some functionality like auditlogging as it uses chronicle packages which needs s390x support. Please note that starting of server as mentioned in Step 3 below is the only verification performed on the binary.

General Notes:

• When following the steps below please use a standard permission user unless otherwise specified.
• A directory /<source_root>/ will be referred to in these instructions, this is a temporary writable directory anywhere you'd like to place it.

Step 1: Build and Install Apache Cassandra

1.1) Build using script

If you want to build Cassandra using manual steps, go to STEP 1.2.

Use the following commands to build Cassandra using the build script. Please make sure you have wget installed.

wget -q https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/ApacheCassandra/4.1.4/build_cassandra.sh

# Build Cassandra
bash build_cassandra.sh   [Provide -j for java to use [Temurin11, OpenJDK11], -t for executing build with tests]

If the build completes successfully, go to STEP 3. In case of error, check logs for more details or go to STEP 1.2 to follow manual build steps.

1.2) Install dependencies

export SOURCE_ROOT=/<source_root>/

• RHEL (7.8, 7.9)

  sudo yum install -y curl git which gcc-c++ make automake autoconf libtool libstdc++-static tar wget patch words libXt-devel libX11-devel texinfo unzip rh-python38
  source /opt/rh/rh-python38/enable

• RHEL (8.6, 8.8, 8.9)

  sudo yum install -y curl git which gcc-c++ make automake autoconf libtool libstdc++ tar wget patch words libXt-devel libX11-devel unzip python39-devel procps gawk maven

• RHEL (9.0, 9.2, 9.3)

  sudo yum install -y curl ant junit ant-junit git which gcc-c++ make automake autoconf libtool libstdc++ tar wget patch words libXt-devel libX11-devel texinfo unzip python3-devel maven procps gawk

• SLES 12 SP5

  sudo zypper install -y curl git which make wget tar zip unzip words gcc7 gcc7-c++ patch libtool automake autoconf ccache xorg-x11-proto-devel xorg-x11-devel alsa-devel cups-devel libffi48-devel libstdc++6-locale glibc-locale libstdc++-devel libXt-devel libX11-devel texinfo gawk gdbm-devel libbz2-devel libdb-4_8-devel libopenssl-devel libuuid-devel readline-devel xz-devel zlib-devel libffi48-devel
  sudo ln -sf /usr/bin/gcc-7 /usr/bin/gcc
  sudo ln -sf /usr/bin/g++-7 /usr/bin/g++
  sudo ln -sf /usr/bin/gcc /usr/bin/cc

• SLES 15 SP5

  sudo zypper install -y gzip curl git which make wget tar zip unzip gcc-c++ patch libtool automake autoconf ccache xorg-x11-proto-devel xorg-x11-devel alsa-devel cups-devel libstdc++6-locale glibc-locale libstdc++-devel libXt-devel libX11-devel texinfo python3-devel maven

• Ubuntu (20.04, 22.04, 23.10)

  sudo apt-get install -y curl ant ant-optional junit git tar g++ make automake autoconf libtool wget patch libx11-dev libxt-dev pkg-config texinfo locales-all unzip python3-dev maven

1.3) Install netty-tcnative

cd $SOURCE_ROOT
wget -q https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/ApacheCassandra/4.1.4/patch/build_netty.sh
bash build_netty.sh -y  # Provide -j for java to use [Temurin11, OpenJDK11]

1.4) Install Java

• With Eclipse Adoptium Temurin Runtime

  • Download and install Eclipse Adoptium Temurin Runtime (Java 11) from here.

• With OpenJDK 11

  • RHEL

    sudo yum install -y java-11-openjdk-devel

  • SLES

    sudo zypper install -y java-11-openjdk

  • Ubuntu

    sudo apt-get install -y openjdk-11-jdk

Note: Version jdk-11.0.20.1+1 was used for Eclipse Adoptium Temurin Runtime while verifying these instructions.

1.5) Set JAVA_HOME

export JAVA_HOME=/<path to java>/
export PATH=$JAVA_HOME/bin:$PATH

1.6) Install maven (Only for RHEL 7.x and SLES 12 SP5)

cd $SOURCE_ROOT
wget https://archive.apache.org/dist/maven/maven-3/3.6.3/binaries/apache-maven-3.6.3-bin.tar.gz
tar -xvzf apache-maven-3.6.3-bin.tar.gz
export PATH=$PATH:$SOURCE_ROOT/apache-maven-3.6.3/bin/

1.7) Build netty

cd $SOURCE_ROOT
git clone https://github.com/netty/netty.git
cd netty
git checkout netty-4.1.58.Final
curl -sSL https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/ApacheCassandra/4.1.4/patch/netty.patch | git apply
mvn install -DskipTests -Dmaven.javadoc.skip=true

1.8) Build Chronicle-bytes

cd $SOURCE_ROOT
git clone https://github.com/OpenHFT/Chronicle-Bytes
cd Chronicle-Bytes
git checkout chronicle-bytes-2.20.111
curl -sSL https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/ApacheCassandra/4.1.4/patch/bytes.patch | git apply
mvn install -DskipTests -Dmaven.javadoc.skip=true

1.9) Install Ant (Only for RHEL 7.x/8.x and SLES)

cd $SOURCE_ROOT
export ANT_HOME=/opt/ant
sudo mkdir -p "$ANT_HOME"
curl -SL -o ant.tar.gz https://archive.apache.org/dist/ant/binaries/apache-ant-1.10.4-bin.tar.gz
sudo tar -xvf ant.tar.gz -C "$ANT_HOME" --strip-components=1
export PATH=$ANT_HOME/bin:$PATH

1.10) Install Python 3.7 (Only for SLES 12.5)

wget https://www.openssl.org/source/openssl-1.1.1q.tar.gz --no-check-certificate
tar -xzf openssl-1.1.1q.tar.gz
cd openssl-1.1.1q
./config --prefix=/usr/local --openssldir=/usr/local
make
sudo make install
sudo ldconfig /usr/local/lib64
export PATH=/usr/local/bin:$PATH

export LDFLAGS="-L/usr/local/lib/ -L/usr/local/lib64/"
export LD_LIBRARY_PATH="/usr/local/lib/:/usr/local/lib64/"
export CPPFLAGS="-I/usr/local/include/ -I/usr/local/include/openssl"

cd $SOURCE_ROOT
wget https://www.python.org/ftp/python/3.7.4/Python-3.7.4.tgz
tar -xzf Python-3.7.4.tgz
cd Python-3.7.4
./configure
make
sudo make install

1.11) Set environment variables

export LANG="en_US.UTF-8"
export JAVA_TOOL_OPTIONS="-Dfile.encoding=UTF8"
export CASSANDRA_USE_JDK11=true
export LD_LIBRARY_PATH=$SOURCE_ROOT/netty-tcnative/boringssl-static/target/native-jar-work/META-INF/native/:$SOURCE_ROOT/netty/transport-native-epoll/target/classes/META-INF/native/

Create the necessary links

sudo ldconfig
sudo ldconfig /usr/local/lib64

1.12) Download Apache Cassandra source code

cd $SOURCE_ROOT/
git clone https://github.com/apache/cassandra.git
cd cassandra
git checkout cassandra-4.1.4
curl -sSL https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/ApacheCassandra/4.1.4/patch/cassandra.patch | git apply
# Use JVM default for stack size
sed -i "/Xss/d" build.xml
sed -i "/Xss/d" conf/jvm-server.options

1.13) Build

cd $SOURCE_ROOT/cassandra
ant

Copy cassandra to /usr/local/

sudo cp -r "$SOURCE_ROOT/cassandra" "/usr/local/"
export PATH=/usr/local/cassandra/bin:$PATH

Step 2: Run test suite (Optional)

cd $SOURCE_ROOT/cassandra/
ant test

Note:
1: If build fails with JavaHeap size issue then set ANT_OPTS="-Xms4G -Xmx4G"

2: In case AuditLoggerTest test fails , delete file ./audit/metadata.cq4t and re-run the test.

3: Several timeout tests and related failures can be addressed by re-running tests after changing the timeout and key_cache_size_in_mb value as follows:

echo "key_cache_size_in_mb: 12" >> "${SOURCE_ROOT}/cassandra/test/conf/cassandra.yaml"
sed -i '/name="test.timeout"/ s/value.*/value="900000" \/>/' "${SOURCE_ROOT}/cassandra/build.xml"

4: To run individual test, use following command:

ant test -Dtest.name=xxx

For example, if org.apache.cassandra.io.sstable.IndexSummaryManagerTest fails, you can execute the individual test by:

ant test -Dtest.name=IndexSummaryManagerTest

Step 3: Start Cassandra Server (Optional)

cassandra -f

Note: In case of an error no permission to create directory /usr/local/cassandra/bin/../data/data, modify the ownership as follows:

   sudo chown -R "$USER" "/usr/local/cassandra"

Start an interactive client as follows in another session after the server is ready

cqlsh
Connected to Test Cluster at 127.0.0.1:9042
[cqlsh 6.1.0 | Cassandra 4.1.4-SNAPSHOT | CQL spec 3.4.6 | Native protocol v5]
Use HELP for help.
cqlsh>

Type in the following commands:

cqlsh> SELECT cluster_name, listen_address FROM system.local;

 cluster_name | listen_address
--------------+----------------
 Test Cluster |     172.18.0.2

(1 rows)

cqlsh> -- Create a keyspace
cqlsh> CREATE KEYSPACE IF NOT EXISTS store WITH REPLICATION = { 'class' : 'SimpleStrategy', 'replication_factor' : '1' };

cqlsh> -- Create a table
cqlsh> CREATE TABLE IF NOT EXISTS store.shopping_cart (
     userid text PRIMARY KEY,
     item_count int,
     last_update_timestamp timestamp
     );

cqlsh> -- Insert some data
cqlsh> INSERT INTO store.shopping_cart
     (userid, item_count, last_update_timestamp)
     VALUES ('9876', 2, toTimeStamp(now()));
cqlsh> INSERT INTO store.shopping_cart
     (userid, item_count, last_update_timestamp)
     VALUES ('1234', 5, toTimeStamp(now()));

cqlsh> SELECT * FROM store.shopping_cart;

 userid | item_count | last_update_timestamp
--------+------------+---------------------------------
   1234 |          5 | 2024-04-02 08:01:10.009000+0000
   9876 |          2 | 2024-04-02 08:01:02.315000+0000

(2 rows)
cqlsh>

If your session looks similar to what's above, your single node cluster is operational!

References:

• http://cassandra.apache.org/
• https://cassandra.apache.org/doc/latest/cassandra/getting_started/index.html
• https://cassandra.apache.org/doc/latest/cassandra/getting_started/querying.html