Skip to content

WiFi firmware brcmfmac crashes when using virtual AP interface #1463

Description

@ingo-h

Describe the bug
For a WiFi router/repeater I use a Raspberry Pi 4B and created an access point with an uplink client connection to the local internet router using a virtual interface of type __ap as created with:

rpi ~$ sudo /sbin/iw dev wlan0 interface add ap@wlan0 type __ap

The router/repeater works stable with 2020-02-13-raspbian-buster-lite. After an upgrade to 2020-05-27-raspios-buster-lite-armhf I noticed from time to time a crash of the brcmfmac firmware, either silently doesn't reply, or with error message

Aug 28 09:04:03 raspberrypi kernel: brcmfmac: brcmf_sdio_hostmail: mailbox indicates firmware halted

Disabling power save does not make a difference with:

rpi ~$ sudo iw dev wlan0 set power_save off

To reproduce
I have made a stress test that associated and disassociated a laptop with a static ip address to the access point in a loop for example with this script using systemd-networkd:

laptop ~$ cat wifistress.sh
#!/bin/bash -e
while true; do
    systemctl start wpa_supplicant@wlan0.service
    sleep 10
    ping -W 5 -c 600 -A 192.168.50.1
    sleep 1
    systemctl stop wpa_supplicant@wlan0.service
    sleep 5
done

192.168.50.1 is the internet router of the uplink on the RasPi so routing is also tested. Calling the script with:

laptop ~$ time sudo ./wifistress.sh

I flashed the image (see above) and run it on the RasPi and setup the testing with:

rpi ~$ sudo -Es
rpi ~# apt update
rpi ~# apt install hostapd

rpi ~# cat > /etc/hostapd/hostapd.conf <<EOF
interface=ap@wlan0
driver=nl80211
ssid=RPiNet
country_code=DE
hw_mode=g
channel=1
auth_algs=1
wpa=2
wpa_passphrase=verySecretPassword
wpa_key_mgmt=WPA-PSK
wpa_pairwise=TKIP
rsn_pairwise=CCMP
EOF

rpi ~# systemctl disable --now dhcpcd.service
rpi ~# rfkill unblock wlan
rpi ~# echo 'allowinterfaces wlan0' >> /etc/dhcpcd.conf

rpi ~# cat > initifs.sh <<EOF
/sbin/iw dev wlan0 interface add ap@wlan0 type __ap
echo 1 > /proc/sys/net/ipv4/ip_forward
iptables -t nat -A POSTROUTING -o wlan0 -j MASQUERADE
/usr/sbin/hostapd -B /etc/hostapd/hostapd.conf
ip addr add 192.168.4.1/24 dev ap@wlan0
EOF

rpi ~# chmod +x initifs.sh

rpi ~# cat > /etc/wpa_supplicant/wpa_supplicant.conf <<EOF
country=DE
ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
update_config=1

network={
    ssid="TestNet"
    psk="verySecretPassword"
    key_mgmt=WPA-PSK
}
EOF

Reboot, then execute:

rpi ~$ sudo systemctl daemon-reload
rpi ~$ sudo ./initifs.sh
rpi ~$ sudo systemctl start dhcpcd.service
rpi ~$ journalctl -f &

On the laptop start the stress test.

Expected behaviour
The stress test from the laptop should run without a crash of the brcmfmac firmware as seen with 2020-02-13-raspbian-buster-lite. There I stopped it after 15 hours without a crash.

Actual behaviour
With 2020-05-27-raspios-buster-lite-armhf the stress test failed after typically 3 to 12 minutes. I haven't seen it running longer than 1 hour.

System
2020-02-13-raspbian-buster-lite comes with firmware-brcm80211_20190114-1+rpt4_all.deb.

2020-05-27-raspios-buster-lite-armhf comes with firmware-brcm80211_20190114-1+rpt6_all.deb.

A workaround to fix the crash is to downgrade to the downloaded firmware-brcm80211_20190114-1+rpt4_all.deb:

rpi ~$ sudo dpkg --purge firmware-brcm80211
rpi ~$ sudo dpkg --install firmware-brcm80211_20190114-1+rpt4_all.deb
rpi ~$ sudo  systemctl reboot

This runs stable all the time as before. The crash is 100 % reproducible when installing the downloaded firmware-brcm80211_20190114-1+rpt6_all again.

Logs
Typical crash:
on the laptop called with time sudo ./wifistress.sh

--- snip ---
64 bytes from 192.168.50.1: icmp_seq=598 ttl=63 time=2.05 ms
64 bytes from 192.168.50.1: icmp_seq=599 ttl=63 time=4.35 ms
64 bytes from 192.168.50.1: icmp_seq=600 ttl=63 time=2.05 ms

--- 192.168.50.1 ping statistics ---
600 packets transmitted, 600 received, 0% packet loss, time 174ms
rtt min/avg/max/mdev = 2.023/3.099/46.393/3.916 ms, pipe 5, ipg/ewma 3.626/2.405 ms
connect: Network is unreachable

real    3m38.373s
user    0m1.055s
sys     0m2.573s
laptop ~$

on the RasPi from the journal:

--- snip ---
Aug 28 09:03:16 raspberrypi hostapd[377]: ap@wlan0: STA 00:17:9a:b9:6e:a9 IEEE 802.11: associated
Aug 28 09:03:16 raspberrypi hostapd[377]: ap@wlan0: STA 00:17:9a:b9:6e:a9 IEEE 802.11: associated
Aug 28 09:03:16 raspberrypi hostapd[377]: ap@wlan0: AP-STA-CONNECTED 00:17:9a:b9:6e:a9
Aug 28 09:03:16 raspberrypi hostapd[377]: ap@wlan0: STA 00:17:9a:b9:6e:a9 RADIUS: starting accounting session F4BE3FBE49C7A46D
Aug 28 09:03:16 raspberrypi hostapd[377]: ap@wlan0: STA 00:17:9a:b9:6e:a9 WPA: pairwise key handshake completed (RSN)
Aug 28 09:03:16 raspberrypi hostapd[377]: ap@wlan0: STA 00:17:9a:b9:6e:a9 RADIUS: starting accounting session F4BE3FBE49C7A46D
Aug 28 09:03:16 raspberrypi hostapd[377]: ap@wlan0: STA 00:17:9a:b9:6e:a9 WPA: pairwise key handshake completed (RSN)
Aug 28 09:03:26 raspberrypi hostapd[377]: ap@wlan0: STA 00:17:9a:b9:6e:a9 IEEE 802.11: disassociated
Aug 28 09:03:26 raspberrypi hostapd[377]: ap@wlan0: AP-STA-DISCONNECTED 00:17:9a:b9:6e:a9
Aug 28 09:03:26 raspberrypi hostapd[377]: ap@wlan0: STA 00:17:9a:b9:6e:a9 IEEE 802.11: disassociated
Aug 28 09:03:26 raspberrypi hostapd[377]: ap@wlan0: STA 00:17:9a:b9:6e:a9 IEEE 802.11: disassociated
Aug 28 09:03:26 raspberrypi hostapd[377]: ap@wlan0: STA 00:17:9a:b9:6e:a9 IEEE 802.11: disassociated
Aug 28 09:03:39 raspberrypi hostapd[377]: ap@wlan0: STA 00:17:9a:b9:6e:a9 IEEE 802.11: associated
Aug 28 09:03:39 raspberrypi hostapd[377]: ap@wlan0: STA 00:17:9a:b9:6e:a9 IEEE 802.11: associated
Aug 28 09:03:39 raspberrypi hostapd[377]: ap@wlan0: AP-STA-CONNECTED 00:17:9a:b9:6e:a9
Aug 28 09:03:39 raspberrypi hostapd[377]: ap@wlan0: STA 00:17:9a:b9:6e:a9 RADIUS: starting accounting session 2E9827C3218501DA
Aug 28 09:03:39 raspberrypi hostapd[377]: ap@wlan0: STA 00:17:9a:b9:6e:a9 WPA: pairwise key handshake completed (RSN)
Aug 28 09:03:39 raspberrypi hostapd[377]: ap@wlan0: STA 00:17:9a:b9:6e:a9 RADIUS: starting accounting session 2E9827C3218501DA
Aug 28 09:03:39 raspberrypi hostapd[377]: ap@wlan0: STA 00:17:9a:b9:6e:a9 WPA: pairwise key handshake completed (RSN)
Aug 28 09:04:03 raspberrypi kernel: brcmfmac: brcmf_sdio_hostmail: mailbox indicates firmware halted

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions