Building Apache Cassandra 3.11

Building Apache Cassandra

The instructions provided below specify the steps to build Apache Cassandra version 3.11.10 on Linux on IBM Z for the following distributions:

• RHEL (7.8, 7.9, 8.1, 8.2, 8.3)
• SLES (12 SP5, 15 SP2)
• Ubuntu (18.04, 20.04, 20.10)

The binary for Apache Cassandra version 3.11.10 can be downloaded from here. Binary works only after replacing snappy-java and jna jars as mentioned in Step 2 and updating java parameters in step 1.6). Please note that starting of server as mentioned in Step 4 below is the only verification performed on the binary.

General Notes:

• When following the steps below please use a standard permission user unless otherwise specified.
• A directory /<source_root>/ will be referred to in these instructions, this is a temporary writable directory anywhere you'd like to place it.

Step 1: Build and Install Apache Cassandra

1.1) Build using script

If you want to build Cassandra using manual steps, go to STEP 1.2.

Use the following commands to build Cassandra using the build script. Please make sure you have wget installed.

wget -q https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/ApacheCassandra/3.11.10/build_cassandra.sh

# Build Cassandra
bash build_cassandra.sh   [Provide -t option for executing build with tests]

If the build completes successfully, go to STEP 4. In case of error, check logs for more details or go to STEP 1.2 to follow manual build steps.

1.2) Install dependencies

export SOURCE_ROOT=/<source_root>/

• RHEL (7.8, 7.9)

  sudo yum install ant junit ant-junit curl git which gcc-c++ make automake autoconf libtool libstdc++-static tar wget patch words libXt-devel libX11-devel texinfo unzip python

  • With OpenJDK

    sudo yum install java-1.8.0-openjdk-devel.s390x

  • With AdoptOpenJDK 8
    • Install AdoptOpenJDK 8 (With Hotspot) from here.

• RHEL (8.1, 8.2, 8.3)

  sudo yum install langpacks-en_GB.noarch curl git which gcc-c++ make automake autoconf libtool libstdc++ tar wget patch words libXt-devel libX11-devel unzip python2

  • With OpenJDK

    sudo yum install java-1.8.0-openjdk-devel.s390x

  • With AdoptOpenJDK 8

    • Install AdoptOpenJDK 8 (With Hotspot) from here.

  • If applicable, link /usr/bin/python to /usr/bin/python2

  sudo ln -sf /usr/bin/python2 /usr/bin/python
  

• SLES 12 SP5

  sudo zypper install --no-recommends ant ant-junit junit curl git which make wget tar zip unzip words gcc-c++ patch libtool automake autoconf ccache xorg-x11-proto-devel xorg-x11-devel alsa-devel cups-devel libffi48-devel libstdc++6-locale glibc-locale libstdc++-devel libXt-devel libX11-devel texinfo python

  • With OpenJDK

    sudo zypper install -y java-1_8_0-openjdk-devel

  • With AdoptOpenJDK 8
    • Install AdoptOpenJDK 8 (With Hotspot) from here.

• SLES (15 SP2)

  sudo zypper install --no-recommends ant junit ant-junit gzip curl git which make wget tar zip unzip gcc-c++ patch libtool automake autoconf ccache  xorg-x11-proto-devel xorg-x11-devel alsa-devel cups-devel libffi-devel libstdc++6-locale glibc-locale libstdc++-devel libXt-devel libX11-devel texinfo python

  • With OpenJDK

  sudo zypper install -y java-1_8_0-openjdk-devel

  • With AdoptOpenJDK 8
    • Install AdoptOpenJDK 8 (With Hotspot) from here.

• Ubuntu (18.04, 20.04, 20.10)

  sudo apt-get update
  sudo apt-get install curl ant ant-optional junit git tar g++ make automake autoconf libtool wget patch libx11-dev libxt-dev pkg-config texinfo locales-all unzip python

  • With OpenJDK

     sudo apt-get install openjdk-8-jre openjdk-8-jdk

  • With AdoptOpenJDK 8
    • Install AdoptOpenJDK 8 (With Hotspot) from here.

1.3) Install Ant (Only for RHEL 8.x)

cd $SOURCE_ROOT/
export ANT_HOME=/opt/ant
sudo mkdir -p "$ANT_HOME"
curl -SL -o ant.tar.gz https://archive.apache.org/dist/ant/binaries/apache-ant-1.10.4-bin.tar.gz
sudo tar -xvf ant.tar.gz -C "$ANT_HOME" --strip-components=1
export PATH=$ANT_HOME/bin:$PATH

1.4) Build Libffi library (Only for SLES, Ubuntu 20.04 and Ubuntu 20.10)

wget ftp://sourceware.org/pub/libffi/libffi-3.2.1.tar.gz
tar xvfz libffi-3.2.1.tar.gz
cd libffi-3.2.1
./configure --prefix=/usr/local
make
sudo make install

1.5) Set environment variables

export LANG="en_US.UTF-8"
export JAVA_TOOL_OPTIONS="-Dfile.encoding=UTF8"
export JAVA_HOME=<path to Java>
export ANT_OPTS="-Xms4G -Xmx4G"
export PATH=$JAVA_HOME/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH #Only for Ubuntu 20.04, 20.10
export LD_LIBRARY_PATH=/usr/local/lib64:$LD_LIBRARY_PATH #Only for SLES

1.6) Download source code

cd $SOURCE_ROOT/
git clone https://github.com/apache/cassandra.git
cd cassandra
git checkout cassandra-3.11.10

1.7) Update build parameters to solve the "StackOverflow" error in execution

sed -i 's/Xss256k/Xss32m/g' build.xml conf/jvm.options

1.8) Build Apache Cassandra:

cd $SOURCE_ROOT/cassandra
ant

Step 2: Replace x86 specific jar files

• Replace Snappy-Java

  cd $SOURCE_ROOT/cassandra
  rm lib/snappy-java-1.1.1.7.jar
  wget -O lib/snappy-java-1.1.2.6.jar https://repo1.maven.org/maven2/org/xerial/snappy/snappy-java/1.1.2.6/snappy-java-1.1.2.6.jar

• Build and replace JNA

  cd $SOURCE_ROOT/
  git clone https://github.com/java-native-access/jna.git
  cd jna
  git checkout 4.2.2
  ant native jar
  rm $SOURCE_ROOT/cassandra/lib/jna-4.2.2.jar
  cp build/jna.jar $SOURCE_ROOT/cassandra/lib/jna-4.2.2.jar

• Copy cassandra to /usr/local/

  sudo cp -r "$SOURCE_ROOT/cassandra" "/usr/local/"
  export PATH=/usr/local/cassandra/bin:$PATH

Step 3: Run test suite (Optional)

cd $SOURCE_ROOT/cassandra/
ant test

• Below mentioned tests fail on s390x as they use testdata which is generated for Little Endian. Hence these failures can be ignored:

  ClientModeSSTableTest
  ScrubTest
  TTLTest
  LegacyLayoutTest
  LegacySSTableTest
  SSTableExportTest
  SASIIndexTest

• Following test suite have intermittent failures

  BatchlogManagerTest
  JsonTest
  ViewTest
  DateTieredCompactionStrategyTest
  CassandraIndexTest
  MigrationCoordinatorTest
  CommitLogSegmentBackpressureTest

• Following test suite can pass with large timeout values

  TimeUUIDTypeTest
  CompactionAwareWriterTest
  AntiCompactionTest
  PartitionImplementationTest
  RangeTest

Note:
1: You may resolve other testcase failures by applying the following changes:

echo "key_cache_size_in_mb: 12" >> "${SOURCE_ROOT}/cassandra/test/conf/cassandra.yaml"
sed -i '/name="test.timeout"/ s/value.*/value="900000" \/>/' "${SOURCE_ROOT}/cassandra/build.xml"

Depending on the test results, you may increase the timeout value by changing the value in the second line. For example: The following statement will change the timeout value to be 4 hours.

sed -i '/name="test.timeout"/ s/value.*/value="14400000" \/>/' "${SOURCE_ROOT}/cassandra/build.xml"

2: In case of failure in org.apache.cassandra.io.sstable.IndexSummaryManagerTest, you can increase the timeout by applying the following changes:

sed -i 's/timeout =.*/timeout = 100000)/' ${SOURCE_ROOT}/cassandra/test/unit/org/apache/cassandra/io/sstable/IndexSummaryManagerTest.java

3: To run individual test, you can execute the following command to execute the test by testname:

ant test -Dtest.name=xxx

For example, if org.apache.cassandra.io.sstable.IndexSummaryManagerTest fails, you can execute the individual test by:

ant test -Dtest.name=IndexSummaryManagerTest

Step 4: Start Cassandra Server(Optional)

cassandra -f

Note: In case of an error no permission to create directory /usr/local/cassandra/bin/../data/data, Run below command.

   sudo chown -R "$USER" "/usr/local/cassandra"

Start an interactive client as follows in another session after the server is ready

cqlsh
Connected to Test Cluster at 127.0.0.1:9042.
[cqlsh 5.0.1 | Cassandra 3.11.10-SNAPSHOT | CQL spec 3.4.4 | Native protocol v4]
Use HELP for help.
cqlsh>

Type in the following commands:

cqlsh> CREATE SCHEMA schema1
    WITH replication = { 'class' : 'SimpleStrategy', 'replication_factor' : 1 };
cqlsh> USE schema1;
cqlsh:schema1> CREATE TABLE users (
                     user_id varchar PRIMARY KEY,
                     first varchar,
                     last varchar,
                     age int
               );
cqlsh:schema1> INSERT INTO users (user_id, first, last, age)
                   VALUES ('jsmith', 'John', 'Smith', 42);
cqlsh:schema1> SELECT * FROM users;

user_id | age | first | last
---------+-----+-------+-------
jsmith |  42 |  John | Smith

(1 rows)
cqlsh:schema1>

If your session looks similar to what's above, congrats, your single node cluster is operational!

References:

http://cassandra.apache.org/
https://cassandra.apache.org/doc/latest/development/testing.html#unit-testing