Building Arrow RS

Building Arrow RS

The instructions provided below specify the steps to build Apache Arrow RS version 59.2.0 on Linux on IBM Z for following distributions:

• RHEL (8.10, 9.6, 9.8, 10.0, 10.2)
• SLES (15 SP7, 16)
• Ubuntu (22.04, 24.04)

General Notes:

• When following the steps below please use a standard permission user unless otherwise specified.

• A directory /<source_root>/ will be referred to in these instructions; by default this is the directory from which the build script is run.

1. Build using script

If you want to build Arrow RS using manual steps, go to Step 2.

Use the following commands to build Arrow RS using the build script. Please make sure you have wget installed.

wget https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/ApacheArrowRS/59.2.0/build_arrow_rs.sh

# Build Arrow RS
bash build_arrow_rs.sh    [Provide -t option for executing build with tests]

In case of error, check logs for more details or go to Step 2 to follow manual build steps.

2. Install Dependencies

export SOURCE_ROOT=/<source_root>/
export PATCH_URL="https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/ApacheArrowRS/59.2.0/patch"

2.1. Install Basic Dependencies

• RHEL 8.10

  sudo yum install -y \
      gcc gcc-c++ make curl git pkgconfig openssl-devel \
      protobuf-compiler lz4-devel libzstd-devel cmake patch \
      python3.12 python3.12-devel ca-certificates
  sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.12 11

• RHEL (9.6, 9.8, 10.0, 10.2)

  sudo yum install -y \
      gcc gcc-c++ make curl git pkgconfig openssl-devel \
      protobuf-compiler lz4-devel libzstd-devel cmake patch ca-certificates

• SLES 15 SP7

  sudo zypper install -y \
      gcc gcc-c++ make curl git pkg-config libopenssl-devel \
      protobuf-devel liblz4-devel libzstd-devel cmake patch \
      python311 python311-devel ca-certificates
  sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.11 11

• SLES 16

  sudo zypper install -y \
      gcc gcc-c++ make curl git pkg-config libopenssl-devel \
      protobuf-devel liblz4-devel libzstd-devel cmake patch ca-certificates

• Ubuntu (22.04, 24.04)

  sudo apt-get update -q
  sudo DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
      build-essential curl git pkg-config libssl-dev \
      protobuf-compiler liblz4-dev libzstd-dev cmake ca-certificates \
      python3 python3-dev

2.2. Install Rust Toolchain

wget -qO- https://sh.rustup.rs | sh -s -- -y --default-toolchain stable
source "$HOME/.cargo/env"
rustup show

3. Build and Install

# Clone the repository
cd $SOURCE_ROOT
git clone --depth 1 --branch 59.2.0 https://github.com/apache/arrow-rs
cd arrow-rs
git submodule update --init --recursive

# Download and apply the s390x patch
wget -q "$PATCH_URL/arrow-rs.patch" -O /tmp/arrow-rs.patch
patch -p1 < /tmp/arrow-rs.patch

# Build
export RUSTFLAGS="-C debuginfo=0"
cargo build --workspace --release

4. Test (Optional)

cd "$SOURCE_ROOT/arrow-rs"
export RUST_BACKTRACE=1
export RUSTFLAGS="-C debuginfo=0"
T="--test-threads=2"

# Phase 1 — Core buffers & schema
cargo test -p arrow-buffer --all-features -- $T
cargo test -p arrow-schema --all-features -- $T

# Phase 2 — Arrow data
cargo test -p arrow-data   --all-features -- $T

# Phase 3 — Array types
cargo test -p arrow-array  --all-features -- $T

# Phase 4 — Arithmetic
cargo test -p arrow-arith  --all-features -- $T

# Phase 5 — IPC
cargo test -p arrow-ipc    --all-features -- $T

# Phase 6 — Compute extensions
cargo test -p arrow-ord     --all-features -- $T
cargo test -p arrow-select  --all-features -- $T
cargo test -p arrow-string  --all-features -- $T
cargo test -p arrow-row     --all-features -- $T
cargo test -p arrow-cast    --all-features -- $T

# Phase 7 — CSV / JSON
cargo test -p arrow-csv  --all-features -- $T
cargo test -p arrow-json --all-features -- $T

# Phase 8 — Avro
cargo test -p arrow-avro -- $T

# Phase 9 — Flight
cargo test -p arrow-flight                -- $T
cargo test -p arrow-flight --all-features -- $T

# Phase 10 — Parquet core
cargo test -p parquet                -- $T
cargo test -p parquet --all-features -- $T

# Phase 11 — Parquet extensions + integration
cargo test -p parquet-variant            -- $T
cargo test -p parquet-variant-json       -- $T
cargo test -p parquet-variant-compute    -- $T
cargo test -p parquet-geospatial         -- $T
cargo test -p arrow-integration-test --all-features -- $T
cargo test -p arrow-integration-testing  -- $T
cargo test -p arrow                      -- $T

5. Verification (Optional)

cd "$SOURCE_ROOT/arrow-rs"
# Confirm the workspace compiled successfully
ls target/release/

To use Arrow RS as a Cargo dependency in your own project:

[dependencies]
arrow = { version = "59.2.0", features = ["..."] }

Notes

This s390x port was completed using AI-assisted tooling with IBM Bob v2.2.0.

References:

• https://github.com/apache/arrow-rs/blob/main/CONTRIBUTING.md#developers-guide-to-arrow-rust