Iperf3 does not work if the server is load-balanced

[uablrek]

Context

• Version of iperf3:
  commit a681713

• Hardware:
  Kvm vm

• Operating system (and distribution, if any):
  Linux 4.19.3

• Other relevant information (for example, non-default compilers,
  libraries, cross-compiling, etc.):

Built on Ubuntu 18.04

Bug Report

• Expected Behavior

It should be possible to setup a number of iperf3 servers behind a load-balancer and use the client with -P option to measure bandwith to multiple servers.

This is not an uncommon measurement case and support it would enhance the usability of iperf3 IMHO. My intention was to measure bandwith towards a "service" in Kubernetes which will load-balance traffic to a number of "pods".

• Actual Behavior

When iperf3 is used towards a load-balanced server the client most often hangs forever.

From tcpdump traces it looks like the iperf3 client actually setup 2 tcp connections, I guess some control/data channel(?). These will be load-balanced to different servers and I think that causes the problem. It works occationally, probably when the 2 tcp connects happen to hit the same server.

• Steps to Reproduce

Simplest lb may be ecmp or multipath routing. I tested this with xcluster

Otherwise any LB solution may be used to see the problem.

• Possible Solution

If possible, re-use the TCP connection rather than setup a new one.

This will not be sufficient for UDP tests where a -u option on the server side whould be the only clean solution I think. You may not want that, so I suggest that UDP is left as-is, TCP is the most important test case.

[bmah888]

This isn't a part of iperf3's design. If this test scenario is really necessary, you might be better served by some other bandwidth measurement tool(s).

[nemethf]

I'd like to mention that iperf2 only opens one connection and therefore works in the described situation.

[ckotil]

iperf2 was a great tip!
But if you want to leverage the improvements in iperf3, then if you can setup affinity on your LB to just client IP, client IP src Port, or client IP and Protocol, then it will work.
e.g. https://cloud.google.com/load-balancing/docs/internal#session_affinity

[SilverSoldier]

Had the same use case, iperf2 did not work for me.
ctraffic seems to handle this and works when the server is behind a load-balancer.

[uablrek]

Cool that you use it 😃 But be aware that iperfX and ctraffic target different test-cases:

Even though ctraffic is pretty fast it is not a performance measurement tool since some bottlenecks are likely in ctraffic itself.

Ctraffic test more how load-balanced traffic is affected by various disturbancies, for instance a reconfiguration or rolling upgrade.