Discrepancy between network behavior in gVisor and runc

[TheQuantumFractal]

Description

Packets that have incorrect MAC destination addresses in their ethernet headers are dropped in runc and gVisor in host network mode. However, in the standard user space networking mode, gVisor does not drop packets with incorrect MAC addresses.

cc: @pawalt, @luiscape

Steps to reproduce

I think you should be able to use scapy python library to create packets to replicate this behavior. You should be able to run this on the host:

from scapy.all import Ether, IP, ICMP, sendp
import time

# Define arbitrary source and destination MAC addresses (that do not correspond to real interfaces)
src_mac = "00:11:22:33:44:55"
dst_mac = "66:77:88:99:AA:BB"

# Define IP addresses for the packet
src_ip = "192.168.1.10"  # Arbitrary source IP address
dst_ip = "192.168.1.1"   # Replace with destination IP address of container

# Create the Ethernet frame with the custom MAC addresses
ether = Ether(src=src_mac, dst=dst_mac)

# Create the IP layer with the source and destination IP addresses
ip = IP(src=src_ip, dst=dst_ip)

# Create the ICMP Echo Request
icmp = ICMP()

# Combine the layers to form the complete packet
packet = ether / ip / icmp

# Send the packet on the network
for _ in range(100):
    sendp(packet, iface="eth0")  # Replace "eth0" with your gVisor container veth
    time.sleep(0.1)

You should be able to run tcpdump -veni eth0 with the container veth on the host to see that response packets are being sent when the request packets should have been dropped in the container. With runc or host networking, no response packets will be received (ICMP request packets are dropped).

runsc version

runsc version 40a09da5a1ab
spec: 1.1.0-rc.1

docker version (if using docker)

No response

uname

Linux ip-10-1-1-1.ec2.internal 5.15.0-209.161.7.2.el9uek.x86_64 #2 SMP Tue Aug 20 10:44:41 PDT 2024 x86_64 x86_64 x86_64 GNU/Linux

kubectl (if using Kubernetes)

No response

repo state (if built from source)

No response

runsc debug logs (if available)

No response

[kevinGC]

We're not verifying incoming MAC addresses, but it should be a quick fix to add that. I assume this never came up because IP routing, in most systems, delivers only the packets destined for the container. If I can ask, is there a reason your containers are receiving errant packets?

[TheQuantumFractal]

Our network stack can change network packets on the host before they reach the container which can cause the container to receive packets with incorrect MAC addresses

[pawalt]

Specifically, we have some custom overlay networking that allows containers to communicate like they're on a private IPV6 subnets between hosts. These containers don't know the MACs of each other, so we realized when working between runc and GVisor that the packets were forwarding fine on GVisor but not runc.