Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Fetching contributors…

Cannot retrieve contributors at this time

67 lines (56 sloc) 1.938 kB
To use RDMA InfiniBand connections the following is needed:
1) The rdma_ucm kernel module must be loaded
2) The application must have read/write access to the IB device nodes
(typically /dev/infiniband/[rdma_cm|uverbs*])
3) The IP address used for the connection must be an address assigned to an
RDMA-capable device (e.g. IPoIB)
4) Shared memory must be sufficient for all RDMA connections,
2 * Global::IATTR_RDMA_RING_BUFFER_SIZE_MB are needed for each connection
set /dev/shm, kernel.shm[min|max|all] accordingly
5) The user must be able to lock the memory registered with verbs, such
that the locked memory limit needs to be sufficient ("ulimit -l" for
bash, "limit memorylocked" for csh). Updating /etc/security/limits.conf
with entries like this is usually adequate (e.g. to raise the limit
to 2GB for all users):
* soft memlock 2048000
* hard memlock 2048000
Ubuntu 11.04 Setup Notes
http://davidhunt.ie/wp/?p=375
Ubuntu 11.04:
blue:~/Software/Equalizer# cat /etc/network/interfaces
auto lo
iface lo inet loopback
auto ib0
iface ib0 inet static
address 192.168.1.1
netmask 255.255.255.0
auto ib1
iface ib1 inet static
address 192.168.1.2
netmask 255.255.255.0
blue:~/Software/Equalizer# cat /etc/udev/rules.d/99-infiniband.rules
KERNEL=="umad*", NAME="infiniband/%k"
KERNEL=="issm*", NAME="infiniband/%k"
KERNEL=="ucm*", NAME="infiniband/%k", MODE="0666"
KERNEL=="uverbs*", NAME="infiniband/%k", MODE="0666"
KERNEL=="ucma", NAME="infiniband/%k", MODE="0666"
KERNEL=="rdma_cm", NAME="infiniband/%k", MODE="0666"
blue:~/Software/Equalizer# cat /etc/modules
# /etc/modules: kernel modules to load at boot time.
#
# This file contains the names of kernel modules that should be loaded
# at boot time, one per line. Lines beginning with "#" are ignored.
lp
rtc
rdma_ucm
rdma_cm
ib_sa
ib_cm
ib_umad
ib_addr
ib_uverbs
ib_ipoib
ib_ipath
ib_qib
mlx4_ib
mlx4_core
Jump to Line
Something went wrong with that request. Please try again.