traffic shaping doesn't work with genie + calico

[ianhe8x]

I want to achieve traffic control with strategies

1. traffic control applies only on connections established initiatively by the container
2. no traffic control between internal ips
3. outside users call rpc, which goes through load-balancer (internal-ip), are consider internal traffic, therefore applies no traffic control

A solution comes to my mind is, with the help of genie, I can create two interface for my pods, the one pod use to visit public network is managed by calico and will respect kubernetes.io/egress-bandwidth, the other one i will use for internal communication (flannel) which will ignore egress-bandwidth. (appreciate helps to improve my solution)

Sadly, I found out with genie installed, kubernetes.io/egress-bandwidth doesn't work any more.
I tested using iperf3. With genie removed, it works again.

[better0332]

@ianhe8x can multus support bandwidth?