RDMA Core Userspace Libraries and Daemons
This is the userspace components for the Linux Kernel's drivers/infiniband subsystem. Specifically this contains the userspace libraries for the following device nodes:
- /dev/infiniband/uverbsX (libibverbs)
- /dev/infiniband/rdma_cm (librdmacm)
- /dev/infiniband/umadX (libibumad)
The userspace component of the libibverbs RDMA kernel drivers are included under the providers/ directory. Support for the following Kernel RDMA drivers is included:
Additional service daemons are provided for:
- srp_daemon (ib_srp.ko)
- iwpmd (for iwarp kernel providers)
- ibacm (for InfiniBand communication management assistant)
This project uses a cmake based build system. Quick start:
$ bash build.sh
build/bin will contain the sample programs and build/lib will contain the shared libraries. The build is configured to run all the programs 'in-place' and cannot be installed.
NOTE: It is not currently easy to run from the build directory, the plugins only load from the system path.
$ apt-get install build-essential cmake gcc libudev-dev libnl-3-dev libnl-route-3-dev ninja-build pkg-config valgrind python3-dev cython3
$ dnf install cmake gcc libnl3-devel libudev-devel pkgconfig valgrind-devel ninja-build python3-devel python3-Cython
NOTE: Fedora Core uses the name 'ninja-build' for the 'ninja' command.
$ zypper install cmake gcc libnl3-devel libudev-devel ninja pkg-config valgrind-devel python3-devel python3-Cython
Building on CentOS 6/7
Install required packages:
$ yum install cmake gcc libnl3-devel libudev-devel make pkgconfig valgrind-devel
Developers on CentOS 7 are suggested to install more modern tooling for the best experience.
$ yum install epel-release $ yum install cmake3 ninja-build pandoc
NOTE: EPEL uses the name 'ninja-build' for the 'ninja' command, and 'cmake3' for the 'cmake' command.
To set up software RDMA on an existing interface with either of the available
driver, use the following commands, substituting
<DRIVER> with the name of the
driver of your choice (
# modprobe <DRIVER> # rdma link add <NAME> type <DRIVER> netdev <DEVICE>
Please note that you need version of
iproute2 recent enough is required for the
command above to work.
You can use either
rdma link to verify that the device was
rxe_cfg is deprecated, please use the
rdma command instead.
Bugs should be reported to the email@example.com mailing list In your bug report, please include:
Information about your system:
- Linux distribution and version
- Linux kernel and version
- InfiniBand hardware and firmware version
- ... any other relevant information
How to reproduce the bug.
If the bug is a crash, the exact output printed out when the crash occurred, including any kernel messages produced.
Patches should also be submitted to the firstname.lastname@example.org mailing list. Please use unified diff form (the -u option to GNU diff), and include a good description of what your patch does and why it should be applied. If your patch fixes a bug, please make sure to describe the bug and how your fix works.
Make sure that your contribution can be licensed under the same license as the original code you are patching, and that you have all necessary permissions to release your work.
Submitted patches must pass the TravisCI automatic builds without warnings. A build similar to TravisCI can be run locally using docker and the 'buildlib/cbuild' script.
$ buildlib/cbuild build-images travis $ buildlib/cbuild pkg travis