Vxlan traffic dropped by default SONIC Drop Rule

[aseaudi]

Description

I created a P2P Vxlan tunnel between 2 sonic switches using the Loopback interfaces as SRC and DST, but it didn't work.

I found out it was because Sonic by default creates iptables rule to DROP INPUT traffic for Loopbacks.

I added a specific allow rule to ACCEPT INPUT DST UDP port 4789, then vxlan traffic worked.

Why does Sonic add those DROP rules ?

Steps to reproduce the issue:

1. Configure OSPF between 2 sonic switches, and ensure Loopback0 is in the routing tables.
2. Create Vlan 10 and add it as untagged egress pvid on a port on each switch. Connect 2 hosts on each switch.
3. Enable ARP Forwarding in ebtables.
4. Configure P2P Vxlan between 2 sonic switches

config vxlan add vtep 172.16.66.1 172.16.66.2
config vxlan evpn_nvo add nvo vtep
config vxlan map add vtep 10 10

config vxlan add vtep 172.16.66.2 172.16.66.1
config vxlan evpn_nvo add nvo vtep
config vxlan map add vtep 10 10

  root@sonic:~# show vxlan interface 
  VTEP Information:
  
      VTEP Name : vtep, SIP  : 172.16.66.1
      NVO Name  : nvo,  VTEP : vtep
      Source interface  : Loopback0
  
  root@sonic:~# show vxlan tunnel
  vxlan tunnel name    source ip    destination ip    tunnel map name    tunnel map mapping(vni -> vlan)
  -------------------  -----------  ----------------  -----------------  ---------------------------------
  vtep                 172.16.66.1  172.16.66.2       map_10_Vlan10      10 -> Vlan10
  root@sonic:~# show vxlan vlan
  +--------+-------+
  | VLAN   |   VNI |
  +========+=======+
  | Vlan10 |    10 |
  +--------+-------+
  Total count : 1
  root@sonic:~# iptables-save
  # Generated by iptables-save v1.8.7 on Sun Jan 16 12:58:32 2022
  *mangle
  :PREROUTING ACCEPT [0:0]
  :INPUT ACCEPT [0:0]
  :FORWARD ACCEPT [0:0]
  :OUTPUT ACCEPT [0:0]
  :POSTROUTING ACCEPT [0:0]
  -A PREROUTING -d 172.16.66.2/32 -p tcp -m tcp --tcp-flags SYN SYN -j TCPMSS --set-mss 1460
  -A PREROUTING -d 172.16.55.1/32 -p tcp -m tcp --tcp-flags SYN SYN -j TCPMSS --set-mss 1460
  -A POSTROUTING -s 172.16.66.2/32 -p tcp -m tcp --tcp-flags SYN SYN -j TCPMSS --set-mss 1460
  -A POSTROUTING -s 172.16.55.1/32 -p tcp -m tcp --tcp-flags SYN SYN -j TCPMSS --set-mss 1460
  COMMIT
  # Completed on Sun Jan 16 12:58:32 2022
  # Generated by iptables-save v1.8.7 on Sun Jan 16 12:58:32 2022
  *filter
  :INPUT ACCEPT [107472:21601742]
  :FORWARD ACCEPT [0:0]
  :OUTPUT ACCEPT [7131427:639555513]
  -A INPUT -s 127.0.0.1/32 -i lo -j ACCEPT
  -A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
  -A INPUT -p icmp -m icmp --icmp-type 8 -j ACCEPT
  -A INPUT -p icmp -m icmp --icmp-type 0 -j ACCEPT
  -A INPUT -p icmp -m icmp --icmp-type 3 -j ACCEPT
  -A INPUT -p icmp -m icmp --icmp-type 11 -j ACCEPT
  -A INPUT -p udp -m udp --dport 67:68 -j ACCEPT
  -A INPUT -p udp -m udp --dport 546:547 -j ACCEPT
  -A INPUT -p tcp -m tcp --dport 179 -j ACCEPT
  -A INPUT -p tcp -m tcp --sport 179 -j ACCEPT
  -A INPUT -d 172.16.66.2/32 -j DROP    <<<< DEFAULT DROP ALL LOOPBACK TRAFFIC ADDED BY SONIC
  -A INPUT -d 172.16.55.1/32 -j DROP
  -A INPUT -d 192.168.11.1/32 -j DROP
  -A INPUT -d 172.16.11.0/32 -j DROP
  -A INPUT -m ttl --ttl-lt 2 -j ACCEPT
  COMMIT
  # Completed on Sun Jan 16 12:58:32 2022
  root@sonic:~#

Describe the results you received:

The vxlan tunnel is created, however we cannot ping over the tunnel

In order to make the tunnel work, I have to add specific INPUT UDP DST PORT 4789 to overcome the default DROP rule created by SONIC for all Loopbacks

root@sonic:~# iptables -I INPUT 10 -p udp -m udp --dport 4789 -j ACCEPT

After that I can ping from host 1 to host 2 over the tunnel

Describe the results you expected:

I should have ping working, and I should not DROP all incoming traffic for the Loopback

Output of show version:

root@sonic:~# show version

SONiC Software Version: SONiC.master.60337-dirty-20211220.183808
Distribution: Debian 11.2
Kernel: 5.10.0-8-2-amd64
Build commit: 0d327abe9
Build date: Mon Dec 20 18:46:04 UTC 2021
Built by: AzDevOps@sonic-build-workers-00101X

Platform: x86_64-kvm_x86_64-r0
HwSKU: Force10-S6000
ASIC: vs
ASIC Count: 1
Serial Number: N/A
Model Number: N/A
Hardware Revision: N/A
Uptime: 13:08:01 up 1 day,  5:25,  1 user,  load average: 0.16, 0.14, 0.18

Docker images:
REPOSITORY                    TAG                                  IMAGE ID       SIZE
docker-dhcp-relay             latest                               a6a3100137c8   436MB
docker-platform-monitor       latest                               34ab0c7b9bce   689MB
docker-platform-monitor       master.60337-dirty-20211220.183808   34ab0c7b9bce   689MB
docker-sonic-mgmt-framework   latest                               f38327843b43   578MB
docker-sonic-mgmt-framework   master.60337-dirty-20211220.183808   f38327843b43   578MB
docker-teamd                  latest                               823108733c9b   436MB
docker-teamd                  master.60337-dirty-20211220.183808   823108733c9b   436MB
docker-syncd-vs               latest                               723cb3fc6cc0   435MB
docker-syncd-vs               master.60337-dirty-20211220.183808   723cb3fc6cc0   435MB
docker-snmp                   latest                               de43b8b8acef   465MB
docker-snmp                   master.60337-dirty-20211220.183808   de43b8b8acef   465MB
docker-sonic-telemetry        latest                               15965d93c8a5   511MB
docker-sonic-telemetry        master.60337-dirty-20211220.183808   15965d93c8a5   511MB
docker-orchagent              latest                               05e9299feb1d   455MB
docker-orchagent              master.60337-dirty-20211220.183808   05e9299feb1d   455MB
docker-sflow                  latest                               a46e3275c303   436MB
docker-sflow                  master.60337-dirty-20211220.183808   a46e3275c303   436MB
docker-router-advertiser      latest                               12cf802b8ea1   423MB
docker-router-advertiser      master.60337-dirty-20211220.183808   12cf802b8ea1   423MB
docker-nat                    latest                               937e9f11accc   438MB
docker-nat                    master.60337-dirty-20211220.183808   937e9f11accc   438MB
docker-lldp                   latest                               ab2869ad3fda   463MB
docker-lldp                   master.60337-dirty-20211220.183808   ab2869ad3fda   463MB
docker-macsec                 latest                               4924107df9a4   439MB
docker-macsec                 master.60337-dirty-20211220.183808   4924107df9a4   439MB
docker-mux                    latest                               55af47e1752b   475MB
docker-mux                    master.60337-dirty-20211220.183808   55af47e1752b   475MB
docker-fpm-frr                latest                               bd2e3744a4b6   454MB
docker-fpm-frr                master.60337-dirty-20211220.183808   bd2e3744a4b6   454MB
docker-gbsyncd-vs             latest                               38f368627c24   431MB
docker-gbsyncd-vs             master.60337-dirty-20211220.183808   38f368627c24   431MB
docker-database               latest                               5b39fa071593   423MB
docker-database               master.60337-dirty-20211220.183808   5b39fa071593   423MB

root@sonic:~# 

Output of show techsupport:

(paste your output here or download and attach the file here )

Additional information you deem important (e.g. issue happens only occasionally):

NOTE: I made a small change in the CLI to enable to add the remote ip address when the Vxlan is created.

[bluecmd]

Note: From discussion on the mailing list, this happens on sonic-vs - not physical hardware. Likely because the VXLAN tunnel is emulated and hits the control plane ACLs.

[zhangyanzhao]

Vxlan is not supported on sonic-vs.