This is an experimenal tool that allows IP communication between two hosts through a shared storage device, which could be a FC or iSCSI LUN mapped to both hosts, or a virtual disk that is attached to two VMs, such as a multi-writer shared disk in VMware vSphere or an EBS Multi-Attach disk in AWS.
It is uncertain what are the the optimal use cases for this tool at present. However, it is possible that it could serve as an additional out-of-band link in high availability (HA) environments to better manage network isolation incidents, or circumvent restrictive firewalls.
It is anticipated that there may be instances of packet drops and other minor interruptions, although low-traffic applications such as SSH are currently functioning properly.
This tool creates a TUN interface and writes incoming packets to a designated disk at a specified block number (wblk
). Simultaneously, it polls the other designated block (rblk
) for changes and writes them to the TUN interface.
host1# posse --disk /dev/sdb --wblk 0 --rblk 1 --addr 10.0.0.1/32 --peer 10.0.0.2/32
host2# posse --disk /dev/sdb --wblk 1 --rblk 0 --addr 10.0.0.2/32 --peer 10.0.0.1/32
host1# ping 10.0.0.2
PING 10.0.0.2 (10.0.0.2) 56(84) bytes of data.
64 bytes from 10.0.0.2: icmp_seq=1 ttl=64 time=136 ms
64 bytes from 10.0.0.2: icmp_seq=2 ttl=64 time=140 ms
64 bytes from 10.0.0.2: icmp_seq=3 ttl=64 time=145 ms
^C
disk
- Path to disk used for sending and receiving packets. It is important to exercise caution when using this disk, as any data on it may be overwritten. Env varDISK
. Required.tun
- Tunnel device name. Env varTUN
. Optional.addr
- Local IP address for tunnel. Must be equal topeer
value on the remote host. Env varADDR
. Required.peer
- Remote IP address for tunnel. Must be equal toaddr
value on the remote host. Env varPEER
. Required.rblk
- Disk block number to read packets from. Must be equal towblk
value on the remote host. Env varRBLK
. Required.wblk
- Disk block number to write packets to. Must be equal torblk
value on the remote host. Env varWBLK
. Required.txqlen
- Transmit queue length. Env varTXQLEN
. Optional. Defaults to 16.rxqlen
- Transmit queue length. Env varRXQLEN
. Optional. Defaults to 16.hz
- Frequency in Hz at which the disk writing and reading operations are performed. Must be equal tohz
value on the remote host. Env varHZ
. Optional. Defaults to 10.promaddr
- Addr:port to listen on for prometheus queries. Env varPROMADDR
. Optional. Defaults to '' (feature disabled).maxstale
- Number of stale reads before declaring peer dead. Env varMAXSTALE
. Optional. Defaults to 5.
The bandwidth is effectively limited by the frequency of transfers, which is set to 10 per second by default (see the hz
flag). This value can be safely increased to 100 on most systems. The maximum feasible value for your configuration can be calculated by dividing 1000 by your shared disk response time in milliseconds. For example, if your disk has a response time of 5 ms, the hz
flag can be set to 200.
It is also recommended to change the TCP congestion control algorithm to BBR on both hosts:
# sysctl -w net.ipv4.tcp_congestion_control=bbr
# sysctl -w net.core.default_qdisc=fq
Posse can transfer data at a rate of approximately 45 kilobytes per second at a frequency of 100 Hertz.
Posse can export metrics to Prometheus. The examples
directory contains a sample configuration files for Prometheus and Grafterm.
Posse can be run in a containerized environment. For more information, please refer to the included Dockerfile and docker-compose.yml files.