RDMA Distribution Driver for Erlang
This is an alternative distribution driver for the Erlang virtual machine. It implements a native protocol for RDMA devices such as InfiniBand. This can potentially improve messaging latency and throughput compared to the default TCP distribution protocol.
This software can be added as a dependency in your project using Rebar or Mix.
It requires OFED to be installed. If it is installed somewhere outside of the
default locations, set the
LDFLAGS environmental variables
appropriately before compiling.
ebin directory must be added to the Erlang VM's code path.
Then you can tell the VM to use the driver by passing the
option. Here is an example:
% erl +K true -pa rdma_dist/ebin -proto_dist rdma -name firstname.lastname@example.org
% iex --erl "+K true -pa rdma_dist/ebin -proto_dist rdma -name email@example.com"
Kernel polling (
+K true) is recommended. If the RDMA device isn't the
default interface on the system, explicitly name it in the
-name option. For
InfiniBand devices, you must have an IP-over-IB device set up---it will be used
for connection management. Once the connection is established, the driver will
use native RDMA verbs.
Tests can be run with
This software has seen limited testing over the following devices:
- Qlogic QDR InfiniBand
- Mellanox QDR InfiniBand
No real-world applications have been tested using this distribution driver. If you have success with other RDMA devices or large Erlang applications, let me know :)