Linux Bridge for IPv4 Broadcast/Multicast Native-POSIX Multiple Instances

[bob2oneil]

I need to create a common Linux bridge type device to attach to multiple virtual Ethernet interfaces for multiple running instances of a Native POSIX Zephyr application. This is for an application that has 2 POSIX virtual Ethernet interfaces. Notionally, this common "bus" represents and over the air simulation of a digital RF channel, where the primary Ethernet interface is the wired input, and the secondary Ethernet interface is the common wireless bus. This is for an IPv4 network only. The firewall is disabled.

My 1st attempt is using the Linux bridge control and slaving a pair of secondary Ethernet interfaces of multiple Zephyr instances. Notionally, the number of nodes on the common bridge will be many. Each running Zephyr application uses a different prefix for the Ethernet interface (e.g. for 3 node network: zeth1.1, zeth2.1, zeth3.1 represent the RF channel to be on a common bus, and zeth1.0, zeth2.0, and zeth3.0 are for the wired interfaces, which are on individual IP subnets and routes and not shared on the network between other primary interfaces). The host side bridge setup occurs prior to the Zephyr instances being executed so that the host side TAP interfaces should be present.

A 2-node example set of commands to establish the Linux bridge are as follows:

        brctl addbr zeth-br
        ip addr add 138.0.0.200/24 dev zeth-br
        brctl addif zeth-br zeth1.1
        brctl addif zeth-br zeth2.1  
        ip link set zeth-br up              
        iptables -I INPUT -i zeth-br -j ACCEPT
        iptables -I FORWARD -i zeth-br -j ACCEPT

I am configuring the TAP interfaces on the Ubuntu host such that the slaved interfaces do not have any assigned IP/subnets/broadcast addresses, while an IP is assigned to the common bridge zeth-br as 138.0.0.200. Each node will transmit on the common OTA channel with a source IP of 138.0.0.[n] where n is the base 1 node number.

My testing results are not good, and while I see packets via Wireshark/Tshark on a slaved virtual Ethernet interfaces, these packet are not viewed on the common interface bridge zeth-br, and hence not received by other network nodes on the zeth-br bridged network.

Also, the IP stack seems to run slower over time, PING times from the host to the wired interface (e.g. zeth1.0) increases, and at some point, no ICMP responses are generated from the application.

Anybody have any success attempting to do this, that is a create a common IP bus/bridge joining together a number of virtual Ethernet interfaces using Native-POSIX emulation?

I have tried quite a few different destination IPs for packets transmitted out of the secondary Ethernet interfaces (zeth1.1, zeth2.1 and zeth3.1) - unicast, multicast, and broadcast - but the packets never appear on the bridge 'zeth-br'.

I did notice that the MAC address of the bridge does match that of the 1st interface added, but this appears to be similar to the test results of other working bridge implementations, where I assume the bridge will take on some of the attributes of the first interface added to it.

Any thoughts on this would be most appreciated.

-----------------------------------------------
Creating OTA Bridge zeth-br
-----------------------------------------------
Found bridge: zeth-br
Executing command brctl addif zeth-br zeth1.1
Executing command brctl addif zeth-br zeth2.1
Bringing up zeth-br

-----------------------------------------------
IP Route Table
-----------------------------------------------
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.200.1   0.0.0.0         UG    600    0        0 wlp3s0
138.0.0.0       0.0.0.0         255.255.255.0   U     0      0        0 zeth-br
169.254.0.0     0.0.0.0         255.255.0.0     U     1000   0        0 wlp3s0
172.17.0.0      0.0.0.0         255.255.0.0     U     0      0        0 docker0
192.0.1.0       0.0.0.0         255.255.255.0   U     0      0        0 zeth1.0
192.0.2.0       0.0.0.0         255.255.255.0   U     0      0        0 zeth2.0
192.168.200.0   0.0.0.0         255.255.255.0   U     600    0        0 wlp3s0

-----------------------------------------------
Zephyr Bridge
-----------------------------------------------
bridge name	bridge id		STP enabled	interfaces
zeth-br		8000.00005e0152ff	no		zeth1.1
							zeth2.1

-----------------------------------------------
Zephyr Bridge MACs
-----------------------------------------------
port no	mac addr		is local?	ageing timer
  1	00:00:5e:01:52:ff	yes		   0.00
  1	00:00:5e:01:52:ff	yes		   0.00
  2	00:00:5e:02:52:ff	yes		   0.00
  2	00:00:5e:02:52:ff	yes		   0.00

-----------------------------------------------
Zephyr Interfaces
-----------------------------------------------
zeth-br: flags=4099<UP,BROADCAST,MULTICAST>  mtu 9000
zeth1.0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1600
zeth1.1: flags=4099<UP,BROADCAST,MULTICAST>  mtu 9000
zeth2.0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1600
zeth2.1: flags=4099<UP,BROADCAST,MULTICAST>  mtu 9000

**ifconfig:**

zeth-br: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 9000
        inet 138.0.0.200  netmask 255.255.255.0  broadcast 0.0.0.0
        inet6 fe80::200:5eff:fe01:52ff  prefixlen 64  scopeid 0x20<link>
        ether 00:00:5e:01:52:ff  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 124  bytes 21519 (21.5 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

zeth1.0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1600
        inet 192.0.1.2  netmask 255.255.255.0  broadcast 192.0.1.255
        inet6 fe80::200:5eff:fe01:53ff  prefixlen 64  scopeid 0x20<link>
        ether 00:00:5e:01:53:ff  txqueuelen 1000  (Ethernet)
        RX packets 1444  bytes 139021 (139.0 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 70640  bytes 16050794 (16.0 MB)
        TX errors 0  dropped 65998 overruns 0  carrier 0  collisions 0

zeth1.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 9000
        inet6 fe80::200:5eff:fe01:52ff  prefixlen 64  scopeid 0x20<link>
        ether 00:00:5e:01:52:ff  txqueuelen 1000  (Ethernet)
        RX packets 69151  bytes 15904730 (15.9 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 121  bytes 21684 (21.6 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

zeth2.0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1600
        inet 192.0.2.2  netmask 255.255.255.0  broadcast 192.0.2.255
        inet6 fe80::200:5eff:fe02:53ff  prefixlen 64  scopeid 0x20<link>
        ether 00:00:5e:02:53:ff  txqueuelen 1000  (Ethernet)
        RX packets 74  bytes 7253 (7.2 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 109  bytes 13668 (13.6 KB)
        TX errors 0  dropped 6 overruns 0  carrier 0  collisions 0

zeth2.1: flags=4099<UP,BROADCAST,MULTICAST>  mtu 9000
        inet6 fe80::200:5eff:fe02:52ff  prefixlen 64  scopeid 0x20<link>
        ether 00:00:5e:02:52:ff  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 46  bytes 8213 (8.2 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

**ip a s:**
zeth-br: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc noqueue state UP group default qlen 1000
    link/ether 00:00:5e:01:52:ff brd ff:ff:ff:ff:ff:ff
    inet 138.0.0.200/24 scope global zeth-br
       valid_lft forever preferred_lft forever
    inet6 fe80::200:5eff:fe01:52ff/64 scope link 
       valid_lft forever preferred_lft forever
48: zeth1.0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1600 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:00:5e:01:53:ff brd ff:ff:ff:ff:ff:ff
    inet 192.0.1.2/24 brd 192.0.1.255 scope global zeth1.0
       valid_lft forever preferred_lft forever
    inet6 fe80::200:5eff:fe01:53ff/64 scope link 
       valid_lft forever preferred_lft forever
49: zeth1.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc fq_codel master zeth-br state UP group default qlen 1000
    link/ether 00:00:5e:01:52:ff brd ff:ff:ff:ff:ff:ff
    inet6 fe80::200:5eff:fe01:52ff/64 scope link 
       valid_lft forever preferred_lft forever
50: zeth2.0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1600 qdisc fq_codel state DOWN group default qlen 1000
    link/ether 00:00:5e:02:53:ff brd ff:ff:ff:ff:ff:ff
    inet 192.0.2.2/24 brd 192.0.2.255 scope global zeth2.0
       valid_lft forever preferred_lft forever
    inet6 fe80::200:5eff:fe02:53ff/64 scope link 
       valid_lft forever preferred_lft forever
51: zeth2.1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 9000 qdisc fq_codel master zeth-br state DOWN group default qlen 1000
    link/ether 00:00:5e:02:52:ff brd ff:ff:ff:ff:ff:ff
    inet6 fe80::200:5eff:fe02:52ff/64 scope link 
       valid_lft forever preferred_lft forever

When using the broadcast destination IP, the traffic originates from one Zephyr node (e.g. zeth1.1) with a source IP of 138.0.0.1 to the broadcast destination 138.0.0.255 using a broadcast destination MAC address. The following is a dump of the tx output from interface zeth1.1 using the TShark application.

 31 280.072001874 fe80::8f4:d4ff:feee:33b1 → ff02::16     ICMPv6 130 Multicast Listener Report Message v2
  32 280.072011483 fe80::200:5eff:fe01:52ff → ff02::16     ICMPv6 110 Multicast Listener Report Message v2
  33 280.307969724 fe80::8f4:d4ff:feee:33b1 → ff02::16     ICMPv6 130 Multicast Listener Report Message v2
  34 280.639963207 fe80::200:5eff:fe01:52ff → ff02::16     ICMPv6 110 Multicast Listener Report Message v2
  35 294.280836802    138.0.0.1 → 138.0.0.255  UDP 230 1 → 49169 Len=188
  36 294.281301383    138.0.0.1 → 138.0.0.255  UDP 230 2 → 49169 Len=188
  37 294.281362775    138.0.0.1 → 138.0.0.255  UDP 230 3 → 49169 Len=188
  38 294.282660905    138.0.0.1 → 138.0.0.255  UDP 230 4 → 49169 Len=188
  39 294.284041718    138.0.0.1 → 138.0.0.255  UDP 230 5 → 49169 Len=188
  40 294.285498792    138.0.0.1 → 138.0.0.255  UDP 230 6 → 49169 Len=188
  41 294.286967721    138.0.0.1 → 138.0.0.255  ECHO 230 Response
  42 294.288357704    138.0.0.1 → 138.0.0.255  UDP 230 8 → 49169 Len=188

Bridge host setup script for two nodes is as follows:

#!/bin/bash
#
# NOTE: bridge support requires "apt-get install bridge-utils"

# Name of over the air bridge
BRIDGE_OTA="zeth-br"

# Output framed banner status for major progress/milestone
separator="-----------------------------------------------"
outputBannerStatus() {
    echo ""
    echo $separator
    echo "$1"
    echo $separator
}

# Check for bridge existence by name
check_bridge() {
    return $(brctl show | awk "/^$1[\t\s]+/{exit 1}")
}

# Remove a bridge by name
remove_bridge() {
    if (! check_bridge "$1"); then
    
        brctl delif $1 zeth1.1
        brctl delif $1 zeth2.1             
        
        echo "Removing bridge: $1"
        ip link set "$1" down
        brctl delbr "$1"
        #iptables -D INPUT -i "$1" -j ACCEPT
        #iptables -D FORWARD -i "$1" -j ACCEPT
    fi
}

if [ $(id -u) != 0 ]; then
    echo "Run this script as a root user!"
    sudo $0 $@
    exit
fi

outputBannerStatus "Starting setup-bridge script..."

# Script verions, update when changes are made
version="Sat 31 Jul 2021 10:56:24 AM EDT"

# Identify build and arguments to script
now="$(date)"
echo "Author: Bob O'Neil"
echo "Current date and time: $now"
echo "Script version: $version"
echo "Script arguments: ${@}"

echo
echo "NOTE: Bridge support requires installation of bridge-utils (apt-get install bridge-utils)"
echo

echo "Removing existing bridge"
remove_bridge $BRIDGE_OTA

echo "Creating bridge $BRIDGE_OTA"
brctl addbr $BRIDGE_OTA
sleep 2

echo "Adding IP address 138.0.0.200 to $BRIDGE_OTA"
ip addr add 138.0.0.200/24 dev $BRIDGE_OTA
sleep 1

echo "Adding interface zeth1.1 to $BRIDGE_OTA"
brctl addif $BRIDGE_OTA zeth1.1
sleep 1

echo "Adding interface zeth2.1 to $BRIDGE_OTA"
brctl addif $BRIDGE_OTA zeth2.1  
sleep 1

echo "Zeroing IP the interfaces"
ifconfig zeth1.1 0.0.0.0
ifconfig zeth2.1 0.0.0.0
sleep 1          

echo "Bringing up interface $BRIDGE_OTA"
ifconfig $BRIDGE_OTA up              
sleep 1

#iptables -I INPUT -i $BRIDGE_OTA -j ACCEPT
#iptables -I FORWARD -i $BRIDGE_OTA -j ACCEPT

# Print Linux "route" table
outputBannerStatus "IP Route Table"
route -n

# Display bridge
outputBannerStatus "Zephyr Bridge (brctl show)"
brctl show $BRIDGE_OTA

outputBannerStatus "Zephyr Bridge STP (brctl showstp)"
brctl showstp $BRIDGE_OTA

# Display bridge MAC addresses
outputBannerStatus "Zephyr Bridge MACs (brctl showmacs)"
brctl showmacs $BRIDGE_OTA

# Display Zephyr interfaces
outputBannerStatus "Zephyr Interfaces (ifconfig | grep zeth)"
ifconfig | grep zeth

# Display IP info
outputBannerStatus "Zephyr Interfaces (ip a s)"
ip a s

# Closing banner
now="$(date)"
outputBannerStatus "Exiting host-setup script on $now"

[bob2oneil]

I now have this working. What appears to have been missing is some bridge net filtering that was solved by adding the following to the configuration script.

    echo "Disabling Bridge Net-Filtering"
    cd /proc/sys/net/bridge
    for f in bridge-nf-*; do echo 0 > $f; done    
    fi