r9700x2-bench

Benchmark suite for a dual AMD Radeon AI PRO R9700 (RDNA4, gfx1201, 32GB each) setup, using llama.cpp's HIP/ROCm backend. Answers concrete questions about this specific hardware topology rather than relying on generic multi-GPU advice:

• Does layer (pipeline) split actually beat row/tensor split on a PCIe-only, 2-hop (GPU→CPU→GPU) topology — or does PCIe 5.0 x16 on both cards compensate enough to make tensor-style splitting worthwhile?
• What's the real dual-GPU vs. single-GPU scaling factor, per model size?
• What's the actual measured P2P bandwidth between the cards (not just link width) — and how does that correlate with the split-mode results?
• How does Flash Attention affect prompt processing at this scale?
• What does power/thermal look like under sustained dual-GPU load?

Hardware / software under test

Component	Spec
GPUs	2x AMD Radeon AI PRO R9700 (RDNA4, gfx1201, 32GB GDDR6 each)
PCIe	Both at PCIe 5.0 x16 (confirmed via /sys/bus/pci/devices/*/current_link_speed)
GPU interconnect	PCIe only, 2 hops (rocm-smi --showtopo) — no switch/fabric bridge
CPU	AMD Ryzen 7 9800X3D (16 threads)
RAM	59GB
OS	Ubuntu 26.04 (Resolute Raccoon)
ROCm	7.1.1

See scripts/collect-sysinfo.sh for the exact commands used to gather all of the above — rerun it on any other machine to compare on equal footing.

Reference / related projects

• pendakwahteknologi/benchmark-rocm — same GPU model (R9700, single-card), same ROCm generation. Best available baseline to diff dual-card numbers against.
• 1337hero/rx7900xtx-llama-bench-rocm — dual-GPU (7900 XTX) llama-bench harness, used as a structural reference for the scripts here.
• ggml-org/llama.cpp — upstream; built from source here since LM Studio only bundles llama-server, not the llama-bench CLI needed for controlled split-mode comparisons.
• ROCm/rocm_bandwidth_test — AMD's official P2P/host bandwidth tool, used to put a real GB/s number on the 2-hop topology.
• ggml-org/llama.cpp multi-GPU docs — background on layer/row/tensor split-mode semantics.

Structure

scripts/
  collect-sysinfo.sh     # dumps GPU/PCIe/ROCm/kernel versions — run first, always
  build-llama-cpp.sh      # clones + builds llama.cpp w/ HIP backend for gfx1201
  bandwidth-test.sh       # builds + runs rocm_bandwidth_test (GPU<->GPU, GPU<->host)
  download-models.sh      # fetches the GGUF models used across the matrix
  run-benchmarks.sh       # orchestrates the full llama-bench matrix, logs power/thermal alongside
  power-monitor.sh        # background rocm-smi sampler, used by run-benchmarks.sh
results/
  <date>-<short-sha>/     # one dir per benchmark run: raw llama-bench output + power log + a summary.md

Quick start

./scripts/collect-sysinfo.sh          # record exact hardware/software state
./scripts/build-llama-cpp.sh          # one-time: build llama-bench for gfx1201
./scripts/bandwidth-test.sh           # one-time: measure real P2P bandwidth
./scripts/download-models.sh          # fetch the model set (see script for sizes/quants)
./scripts/run-benchmarks.sh           # run the full matrix, write results/<date>/

Each script is standalone and safe to re-run; run-benchmarks.sh is the only long-running one (expect it to take a while across the full model × split-mode × flash-attention matrix).