note this is proof-of-concept project, do not use this in production
This is a prototype for a L3-DSR hash based load balancer. For a client I was researching if we could improve its CDN (for streaming content) by using a hash-based loadbalancer with the capability of using Layer-3 based Direct Server Return (so that returning traffic is bypassing the loadbalancer).
This codebase only implements a naive balancer application which:
- creates a handshake with the client
- inspects the first packet to determine which server to route the traffic to
- syncs the TCP handshake with the packetbridge application
- forwards all further TCP traffic to the packetbridge
By using the DSCP field in the IPv4 header, the loadbalancer identifies itself to the packetbride.
The packetbridge application:
- receives TCP packets from the balancer application
- creates (for new connections) a handshake with the backend (e.g. NGINX)
- modifies the incoming traffic so that it matches with the TCP handshake of the backend
- modifies outgoing traffic so that it matches with the TCP handshake of the client.
The packetbride will use the loadbalancer IP address (it knows because of the DSCP field) for outgoing traffic, so that outgoing traffic bypasses the loadbalancer.
The easiest way to play with this project is to setup a Vagrant environment.
Running vagrant up
will setup two boxes, one for the balancer, the other
for the backend.
Note that all cli argument have defaults that matches the Vagrant environment.
Run the following commands to start the balancer (within the Vagrant environment).
vagrant ssh balancer
cd src/github.com/brocaar/l3dsr-hash-balancer
go get ./...
make
sudo ./bin/balancer
The balancer box has one interface 192.168.33.10
on which it listens for
incoming requests.
Run the following commands to start the packetbridge (within the Vagrant environment).
vagrant ssh backend
cd src/github.com/brocaar/l3dsr-hash-balancer
sudo ./bin/packetbridge
The backend box has two interfaces. On 192.168.33.20
it listens for incoming
packets from the balancer. On 192.168.33.30
NGINX is running.
Now that both applications are running, you can make a request to
http://192.168.33.10/
. This will:
- Create a TCP handshake between you and the balancer (
.10
) - The balancer (
.10
) will sync your TCP handshake with the packetbridge (.20
) - The packetbridge (
.20
) will create a TCP handshake with the backend (NGINX on.30
). - The packetbridge will start forwarding your packets to NGIXN (
.30
) and the packets from NGINX to you (by using the.10
source ip). Your HTTP client will think that all packets came from the balancer :-)