Skip to content

Added rocm lib path discovery with TheRock support#227

Merged
gilbertlee-amd merged 1 commit intoROCm:developfrom
gahan9:therock_support
Jan 8, 2026
Merged

Added rocm lib path discovery with TheRock support#227
gilbertlee-amd merged 1 commit intoROCm:developfrom
gahan9:therock_support

Conversation

@gahan9
Copy link
Copy Markdown
Contributor

@gahan9 gahan9 commented Jan 8, 2026

Motivation

Compilation failure with https://github.com/ROCm/TheRock installation

Technical Details

  1. Added path for device libs as to either of generic path $(ROCM_PATH)/amdgcn/bitcode or $(ROCM_PATH)/lib/llvm/amdgcn/bitcode)
  2. Replaces Transferbench compilation argument from $(HIPCC) to $(CXX) to comply with overridden CXX compiler environment

Test Plan

  1. Install amdgpu driver
  2. Install TheRock nightly
  3. Install openmpi-bin, openmpi-common and libopenmpi-dev
  4. Clone transferbench
  5. Modify MPI_PATH to where include/mpi.h exists
  6. Modify ROCM_PATH which points to TheRock stack

Test Result

Build successful and execution working as expected:

$ make
Makefile:42: "Could not find /opt/rocm-7.11.0/bin/amdclang++. Using fallback to /opt/rocm-7.11.0/bin/hipcc"
Building with NIC executor support. Can set DISABLE_NIC_EXEC=1 to disable
Building with MPI communicator support.  Can set DISABLE_MPI_COMM=1 to disable
/opt/rocm-7.11.0/bin/hipcc -I/opt/rocm-7.11.0/include -I/opt/rocm-7.11.0/include/hip -I/opt/rocm-7.11.0/include/hsa -DNIC_EXEC_ENABLED -DMPI_COMM_ENABLED -I/usr/lib/x86_64-linux-gnu/openmpi/include -Wall -x hip -D__HIP_PLATFORM_AMD__ -D__HIPCC__ "--offload-arch=native" --rocm-device-lib-path=/opt/rocm-7.11.0/lib/llvm/amdgcn/bitcode -O3 -I./src/header -I./src/client -I./src/client/Presets src/client/Client.cpp -o TransferBench -lnuma -L/opt/rocm-7.11.0/lib -lhsa-runtime64 -lamdhip64 -lpthread -libverbs -L//usr/lib/x86_64-linux-gnu/openmpi/lib -lmpi

$ ./TransferBench 
TransferBench v1.66.00 (with MPI+NIC support) (Single-node mode)
=============================================================================================================
Usage: ./TransferBench config <N>
  config: Either:
          - Filename of configFile containing Transfers to execute (see example.cfg for format)
          - Name of preset config:
  N     : (Optional) Number of bytes to copy per Transfer.
          If not specified, defaults to 268435456 bytes. Must be a multiple of 4 bytes
          If 0 is specified, a range of Ns will be benchmarked
          May append a suffix ('K', 'M', 'G') for kilobytes / megabytes / gigabytes

Environment variables:
======================
 ALWAYS_VALIDATE   - Validate after each iteration instead of once after all iterations
...
...
...

Submission Checklist

Signed-off-by: Gahan Saraiya <Gahan.Saraiya@amd.com>
@gahan9 gahan9 requested a review from a team as a code owner January 8, 2026 06:14
@gilbertlee-amd gilbertlee-amd merged commit eb96663 into ROCm:develop Jan 8, 2026
7 checks passed
@gahan9 gahan9 deleted the therock_support branch January 9, 2026 04:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants