OVN: QoS max bandwidth / burst only works if your packet will travel via tunnel

[mangelajo]

This patch (2 years ago) introduces a regression in combination with distributed floating ip, or direct port attachment to provider network.

openvswitch/ovs@a6095f8

If you have a port which is directly attached to a provider network, or that is configured with a distributed floating ip, the packets will never travel through the tunnel port, and the traffic won't be limited.

[alissonpmedeiros]

Hi @mangelajo

When I'm using OVN with docker in “overlay” mode, the docker network(with global visibility) is used to connect containers in local OVS that are controlled by local ovn-controller and docker is responsible to provide floating ip.

I'm thinking the packets are not traveling through tunnel port and can't limit max bandwidth / burst.

Steps to apply qos:
Step 1
ovn-nbctl list Logical_Switch_Port

Step 2
Get _uuid from Logical_Switch_Port table

Step 3
ovn-nbctl set Logical_Switch_Port _uuid options:qos_max_rate=1000
ovn-nbctl set Logical_Switch_Port _uuid options:qos_burst=100

Step 4
ovn-nbctl list Logical_Switch_Port

_uuid : 1f966b30-1afe-47fa-ad2e-e79b1175d56e
addresses : ["02:93:90:fb:ac:b2 192.168.200.2"]
dhcpv4_options : []
dhcpv6_options : []
dynamic_addresses : []
enabled : []
external_ids : {}
name : "ebfeb03102ac7c2e903ae03e0d0689537a3962bdfdfebecae9f2c65c2ec14422"
options : {qos_burst="100", qos_max_rate="1000"}
parent_name : []
port_security : []
tag : []
tag_request : []
type : ""
up : true

_uuid : ea5e6a33-3a31-4341-ab4e-d2a7fc814b73
addresses : ["02:18:36:d5:6c:05 192.168.200.3"]
dhcpv4_options : []
dhcpv6_options : []
dynamic_addresses : []
enabled : []
external_ids : {}
name : "a7d8bc402c1fb33e09756a5a6333a6de71f5d2c0bd61fec19b5bef7e59b1c942"
options : {}
parent_name : []
port_security : []
tag : []
tag_request : []
type : ""
up : true

Do you have any idea how to solve this problem?

I'm following this tutorial.

[mangelajo]

This needs to be fixed in ovn-controller, it has to create a queue on the network interface associated to the provider network bridge.

I'm unsure by your description of your exact topology.

Can you do:

$ ovs-vsctl list Open .

And show me what you have ?

and a

$ ovs-vsctl show

[alissonpmedeiros]

In DC1(hostname) with IP 10.7.229.135:
ovs-vsctl show

d2695398-ca9e-46ea-800d-5b358a25595d
Bridge br-int
Port br-int
Interface br-int
type: internal
Port "ovn-46f301-0"
Interface "ovn-46f301-0"
type: geneve
options: {csum="true", key=flow, remote_ip="10.7.229.141"}
Port "ebfeb03102ac7c2"
Interface "ebfeb03102ac7c2"
ovs_version: "2.8.0"

ovs-vsctl list Open .

_uuid : d2695398-ca9e-46ea-800d-5b358a25595d
bridges : [ec546e22-5798-4ec3-9af9-59ed5dfbc61e]
cur_cfg : 285
datapath_types : [netdev, system]
db_version : "7.15.0"
external_ids : {hostname="dc1", ovn-bridge=br-int, ovn-encap-ip="10.7.229.135", ovn-encap-type=geneve, ovn-nb="tcp:10.7.229.138:6641", ovn-remote="tcp:10.7.229.138:6642", rundir="/var/run/openvswitch", system-id="1b2fdcef-48cc-419f-9d9d-f6a025d9f2dc"}
iface_types : [geneve, gre, internal, lisp, patch, stt, system, tap, vxlan]
manager_options : []
next_cfg : 285
other_config : {}
ovs_version : "2.8.0"
ssl : []
statistics : {}
system_type : centos
system_version : "7"

In DC2(hostname) with IP 10.7.229.141:
ovs-vsctl show

9a7bead6-e578-4c45-ad8d-aca506ed4c48
Bridge br-int
fail_mode: secure
Port "a7d8bc402c1fb33"
Interface "a7d8bc402c1fb33"
Port "ovn-1b2fdc-0"
Interface "ovn-1b2fdc-0"
type: geneve
options: {csum="true", key=flow, remote_ip="10.7.229.135"}
Port br-int
Interface br-int
type: internal
ovs_version: "2.8.0"

ovs-vsctl list Open .

_uuid : 9a7bead6-e578-4c45-ad8d-aca506ed4c48
bridges : [14d32fd5-9d06-4271-98a6-bee9d1efe317]
cur_cfg : 196
datapath_types : [netdev, system]
db_version : "7.15.0"
external_ids : {hostname="dc2", ovn-bridge=br-int, ovn-encap-ip="10.7.229.141", ovn-encap-type=geneve, ovn-nb="tcp:10.7.229.138:6641", ovn-remote="tcp:10.7.229.138:6642", rundir="/var/run/openvswitch", system-id="46f3010f-46b9-4d8b-86ac-0ad1291de1b8"}
iface_types : [geneve, gre, internal, lisp, patch, stt, system, tap, vxlan]
manager_options : []
next_cfg : 196
other_config : {}
ovs_version : "2.8.0"
ssl : []
statistics : {}
system_type : centos
system_version : "7"

Each VM has one container attached in local OVS and they can communicate each other via geneve tunnel. The local OVS has two ports, one to remote IP and another to container

[alissonpmedeiros]

Logical topology(OVN-Northbound):
ovn-nbctl show

switch 4a2f3caf-5ddf-4cba-b606-f4be7dbb2f24 (e02f407e6cbced180871ac0a60da7a83fe9225c4674f72b8abdd8909d708a3c2)
port ebfeb03102ac7c2e903ae03e0d0689537a3962bdfdfebecae9f2c65c2ec14422
addresses: ["02:93:90:fb:ac:b2 192.168.200.2"]
port a7d8bc402c1fb33e09756a5a6333a6de71f5d2c0bd61fec19b5bef7e59b1c942
addresses: ["02:18:36:d5:6c:05 192.168.200.3"]

Physical topology(OVN-Southbound):
ovn-sbctl show

Chassis "1b2fdcef-48cc-419f-9d9d-f6a025d9f2dc"
hostname: "dc1"
Encap geneve
ip: "10.7.229.135"
options: {csum="true"}
Port_Binding "ebfeb03102ac7c2e903ae03e0d0689537a3962bdfdfebecae9f2c65c2ec14422"
Chassis "46f3010f-46b9-4d8b-86ac-0ad1291de1b8"
hostname: "dc2"
Encap geneve
ip: "10.7.229.141"
options: {csum="true"}
Port_Binding "a7d8bc402c1fb33e09756a5a6333a6de71f5d2c0bd61fec19b5bef7e59b1c942"

[alissonpmedeiros]

When I execute ping from container(192.168.200.2) in DC1(10.7.229.135) to container(192.168.200.3) in DC2(10.7.229.141), tcpdump in DC1 show:
tcpdump -i eth0 | grep Geneve

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes

14:06:16.591332 IP dc1.20350 > 10.7.229.141.6081: Geneve, Flags [C], vni 0x1, options [8 bytes]: IP 192.168.200.2 > 192.168.200.3: ICMP echo request, id 24, seq 144, length 64

14:06:16.593204 IP 10.7.229.141.61400 > dc1.6081: Geneve, Flags [C], vni 0x1, options [8 bytes]: IP 192.168.200.3 > 192.168.200.2: ICMP echo reply, id 24, seq 144, length 64

14:06:17.207917 IP 10.7.229.141.61400 > dc1.6081: Geneve, Flags [C], vni 0x1, options [8 bytes]: IP 192.168.200.3 > 192.168.200.2: ICMP echo request, id 25, seq 140, length 64

[syswu]

I encounter the same problem, how can I fix it?

---

[root@worker1 ~]# ovs-vsctl --version
ovs-vsctl (Open vSwitch) 2.11.0
DB Schema 7.16.1