FS#3989 - MT76x2E randomly restarts on Xiaomi Mi Router 4A Gigabit Edition

[openwrt-bot]

aleatoricLilac:

Device: Xiaomi Mi Router 4A Gigabit Edition (MT7612)
OpenWrt version: 21.02.0-rc4 r16256-2d5ee43dc6

5GHz wireless appears to encounter message timeouts and restarts seemingly randomly; dmesg is as follows:

[19753.943374] mt76x2e 0000:01:00.0: MCU message 1f (seq 11) timed out
[19753.988543] mt76x2e 0000:01:00.0: Firmware Version: 0.0.00
[19753.988561] mt76x2e 0000:01:00.0: Build: 1
[19753.988569] mt76x2e 0000:01:00.0: Build Time: 201507311614____
[19754.003378] mt76x2e 0000:01:00.0: Firmware running!
[19754.007557] ieee80211 phy1: Hardware restart was requested

[25595.538192] mt76x2e 0000:01:00.0: MCU message 1f (seq 12) timed out
[25595.583343] mt76x2e 0000:01:00.0: Firmware Version: 0.0.00
[25595.583360] mt76x2e 0000:01:00.0: Build: 1
[25595.583368] mt76x2e 0000:01:00.0: Build Time: 201507311614____
[25595.598154] mt76x2e 0000:01:00.0: Firmware running!
[25595.602399] ieee80211 phy1: Hardware restart was requested

Configuration is a single SSID in AP mode with no changes to default settings other than security set to WPA2-PSK/CCMP (AES).

[openwrt-bot]

gstammw:

Bug confirmed on OpenWrt 21.02.0 FINAL

[76474.759778] ieee80211 phy1: Hardware restart was requested
[79380.111752] mt76x2e 0000:01:00.0: MCU message 1f (seq 7) timed out
[79380.156926] mt76x2e 0000:01:00.0: Firmware Version: 0.0.00
[79380.162432] mt76x2e 0000:01:00.0: Build: 1
[79380.166620] mt76x2e 0000:01:00.0: Build Time: 201507311614____
[79380.187734] mt76x2e 0000:01:00.0: Firmware running!
[79380.195918] ieee80211 phy1: Hardware restart was requested
[79390.408570] mt76x2e 0000:01:00.0: Firmware Version: 0.0.00
[79390.414081] mt76x2e 0000:01:00.0: Build: 1
[79390.418232] mt76x2e 0000:01:00.0: Build Time: 201507311614____
[79390.439410] mt76x2e 0000:01:00.0: Firmware running!
[79390.447576] ieee80211 phy1: Hardware restart was requested
[80761.349345] mt76x2e 0000:01:00.0: Firmware Version: 0.0.00
[80761.354896] mt76x2e 0000:01:00.0: Build: 1
[80761.359040] mt76x2e 0000:01:00.0: Build Time: 201507311614____
[80761.380167] mt76x2e 0000:01:00.0: Firmware running!


[openwrt-bot]

gormih:

[ 0.000000] Linux version 5.4.154 (builder@buildhost) (gcc version 8.4.0 (OpenWrt GCC 8.4.0 r16325-88151b8303)) #0 SMP Sun Oct 24 09:01:35 2021
[ 0.000000] SoC Type: MediaTek MT7621 ver:1 eco:3
[ 0.000000] printk: bootconsole [early0] enabled
[ 0.000000] CPU0 revision is: 0001992f (MIPS 1004Kc)
[ 0.000000] MIPS: machine is Xiaomi Mi Router 4A Gigabit Edition

......

[90427.178591] mt76x2e 0000:01:00.0: Firmware running!
[90427.186750] ieee80211 phy1: Hardware restart was requested
[90502.472814] mt76x2e 0000:01:00.0: Firmware Version: 0.0.00
[90502.478324] mt76x2e 0000:01:00.0: Build: 1
[90502.482483] mt76x2e 0000:01:00.0: Build Time: 201507311614____
[90502.503650] mt76x2e 0000:01:00.0: Firmware running!
[90502.511806] ieee80211 phy1: Hardware restart was requested
[90534.643543] mt76x2e 0000:01:00.0: Firmware Version: 0.0.00
[90534.649050] mt76x2e 0000:01:00.0: Build: 1
[90534.653204] mt76x2e 0000:01:00.0: Build Time: 201507311614____
[90534.674385] mt76x2e 0000:01:00.0: Firmware running!
[90534.682627] ieee80211 phy1: Hardware restart was requested
[90565.666303] mt76x2e 0000:01:00.0: Firmware Version: 0.0.00
[90565.671808] mt76x2e 0000:01:00.0: Build: 1
[90565.675994] mt76x2e 0000:01:00.0: Build Time: 201507311614____
[90565.697191] mt76x2e 0000:01:00.0: Firmware running!
[90565.705338] ieee80211 phy1: Hardware restart was requested
[90599.081026] mt76x2e 0000:01:00.0: Firmware Version: 0.0.00
[90599.086603] mt76x2e 0000:01:00.0: Build: 1
[90599.090752] mt76x2e 0000:01:00.0: Build Time: 201507311614____
[90599.111856] mt76x2e 0000:01:00.0: Firmware running!
[90599.120046] ieee80211 phy1: Hardware restart was requested
[90650.387011] mt76x2e 0000:01:00.0: Firmware Version: 0.0.00
[90650.392517] mt76x2e 0000:01:00.0: Build: 1
[90650.396694] mt76x2e 0000:01:00.0: Build Time: 201507311614____
[90650.417850] mt76x2e 0000:01:00.0: Firmware running!
[90650.426075] ieee80211 phy1: Hardware restart was requested
[90777.502069] mt76x2e 0000:01:00.0: Firmware Version: 0.0.00
[90777.507598] mt76x2e 0000:01:00.0: Build: 1
[90777.511748] mt76x2e 0000:01:00.0: Build Time: 201507311614____
[90777.532960] mt76x2e 0000:01:00.0: Firmware running!
[90777.541109] ieee80211 phy1: Hardware restart was requested

[openwrt-bot]

laigor:

Logged kernel panic:
[Mon Jan 31 17:53:19 2022] ieee80211 phy1: Hardware restart was requested
[Mon Jan 31 17:53:24 2022] mt76x2e 0000:01:00.0: MCU message 1f (seq 12) timed out
[Mon Jan 31 17:53:24 2022] mt76x2e 0000:01:00.0: Firmware Version: 0.0.00
[Mon Jan 31 17:53:24 2022] mt76x2e 0000:01:00.0: Build: 1
[Mon Jan 31 17:53:24 2022] mt76x2e 0000:01:00.0: Build Time: 201507311614____
[Mon Jan 31 17:53:24 2022] mt76x2e 0000:01:00.0: Firmware running!
[Mon Jan 31 17:53:24 2022] ieee80211 phy1: Hardware restart was requested
[Mon Jan 31 17:53:30 2022] mt76x2e 0000:01:00.0: MCU message 1f (seq 14) timed out
[Mon Jan 31 17:53:30 2022] mt76x2e 0000:01:00.0: Firmware Version: 0.0.00
[Mon Jan 31 17:53:30 2022] mt76x2e 0000:01:00.0: Build: 1
[Mon Jan 31 17:53:30 2022] mt76x2e 0000:01:00.0: Build Time: 201507311614____
[Mon Jan 31 17:53:30 2022] mt76x2e 0000:01:00.0: Firmware running!
[Mon Jan 31 17:53:30 2022] ieee80211 phy1: Hardware restart was requested
[Mon Jan 31 17:53:31 2022] mt76x2e 0000:01:00.0: MCU message 02 (seq 2) timed out
[Mon Jan 31 17:53:32 2022] mt76x2e 0000:01:00.0: MCU message 1f (seq 3) timed out
[Mon Jan 31 17:53:32 2022] mt76x2e 0000:01:00.0: Firmware Version: 0.0.00
[Mon Jan 31 17:53:32 2022] mt76x2e 0000:01:00.0: Build: 1
[Mon Jan 31 17:53:32 2022] mt76x2e 0000:01:00.0: Build Time: 201507311614____
[Mon Jan 31 17:53:32 2022] mt76x2e 0000:01:00.0: Firmware running!
[Mon Jan 31 17:53:32 2022] ieee80211 phy1: Hardware restart was requested
[Mon Jan 31 17:53:37 2022] mt76x2e 0000:01:00.0: MCU message 1f (seq 4) timed out
[Mon Jan 31 17:53:37 2022] mt76x2e 0000:01:00.0: Firmware Version: 0.0.00
[Mon Jan 31 17:53:37 2022] mt76x2e 0000:01:00.0: Build: 1
[Mon Jan 31 17:53:37 2022] mt76x2e 0000:01:00.0: Build Time: 201507311614____
[Mon Jan 31 17:53:37 2022] mt76x2e 0000:01:00.0: Firmware running!
[Mon Jan 31 17:53:37 2022] ieee80211 phy1: Hardware restart was requested
.........
[Mon Jan 31 17:55:10 2022] skbuff: skb_under_panic: text:285d9eda len:66 put:2 head:2904025b data:834a74a9 tail:0x85079c40 end:0x85079f00 dev:wlan1
[Mon Jan 31 17:55:10 2022] Kernel bug detected[#1]:
[Mon Jan 31 17:55:10 2022] CPU: 2 PID: 872 Comm: mt76-tx phy1 Tainted: G W 5.4.171 #0
[Mon Jan 31 17:55:10 2022] $ 0 : 00000000 00000001 00000078 00000000
[Mon Jan 31 17:55:10 2022] $ 4 : 8101c34c 8101c34c 81021070 00000000
[Mon Jan 31 17:55:10 2022] $ 8 : 316e616c 00000414 66393730 64203030


[openwrt-bot]

laigor:

I compile 21.02 with fresh mt76 dated 2022-01-31 and the problem is gone.

[untuonac]

Hi,
my Xiaomi MI 4A Gigabit with OpenWrt 22.03.0-rc6 r19590-042d558536 also has the problem and I can't solve.
Do you have suggestions or news?

[dvalter]

Both official 21.02.5 and official 22.03.2 are broken.

But as mentioned above, building with mt76 from around 2022-01-31 likely fixes this bug. At least my 4a is fine for 6+ days whereas the previous best was around 3 days before the bug occurrence.
I took 2022-02-03 and reverted all offending commits, feel free to use the patch and write here whether the bug does appear for you.

[xfrv]

Both official 21.02.5 and official 22.03.2 are broken.

But as mentioned above, building with mt76 from around 2022-01-31 likely fixes this bug. At least my 4a is fine for 6+ days whereas the previous best was around 3 days before the bug occurrence. I took 2022-02-03 and reverted all offending commits, feel free to use the patch and write here whether the bug does appear for you.

Hey!, do you know how to apply this patch?
Thank you

[dvalter]

Hey!, do you know how to apply this patch? Thank you

Follow the guide. Use v21.02.5 instead of v21.02.3.
Apply the patch with curl -sLf https://gist.github.com/dvalter/a9fe454e3ff7c5e32ed1df2b841eec99/raw/6d0566f1fc8a0b4bacfe24d5a1ec2decd97b8bed/openwrt-2102.patch | patch -p1 right after the git checkout command.

In menuconfig you may want to enable luci -> collections -> luci as well as more non-essential packages if you need then.

[xfrv]

Hey!, do you know how to apply this patch? Thank you

Follow the guide. Use v21.02.5 instead of v21.02.3. Apply the patch with curl -sLf https://gist.github.com/dvalter/a9fe454e3ff7c5e32ed1df2b841eec99/raw/6d0566f1fc8a0b4bacfe24d5a1ec2decd97b8bed/openwrt-2102.patch | patch -p1 right after the git checkout command.

In menuconfig you may want to enable luci -> collections -> luci as well as more non-essential packages if you need then.

Thank you very much for helping me, the patch is working for me!

[webysther]

This patch is needed for 22.03?

[fehead]

I am getting the same error on 2.4G wifi.
Model : NETIS WF2881, ramips/mt7621
Version : OpenWrt 22.03.5 r20134-5f15225c1e
Kernel version : 5.10.176

[ 93.179112] mt76x2e 0000:02:00.0: MCU message 02 (seq 15) timed out
[ 93.290254] mt76x2e 0000:02:00.0: Firmware Version: 0.0.00
[ 93.301213] mt76x2e 0000:02:00.0: Build: 1
[ 93.309423] mt76x2e 0000:02:00.0: Build Time: 201607111443____
[ 93.324469] mt76x2e 0000:02:00.0: Firmware running!
[ 93.339422] ieee80211 phy1: Hardware restart was requested
[ 94.379075] mt76x2e 0000:02:00.0: MCU message 02 (seq 3) timed out
[ 94.500112] mt76x2e 0000:02:00.0: Firmware Version: 0.0.00
[ 94.511075] mt76x2e 0000:02:00.0: Build: 1
[ 94.519285] mt76x2e 0000:02:00.0: Build Time: 201607111443____
[ 94.534326] mt76x2e 0000:02:00.0: Firmware running!
[ 94.549080] ieee80211 phy1: Hardware restart was requested

[webysther]

I got fixed creating a one 2,4/5Ghz with SSID using the same name and turning off the DNS in my network. I don't know what be the root cause, but maybe help anyone.

[dvalter]

@webysther does it fix the problem of 5Ghz breaking, or just helps clients to downgrade to 2.4 and keep going?

[Neustradamus]

To follow

[webysther]

@dvalter fix the problem to me, try using the same SSID for both.

[fehead]

I tried switching the 2.4G and 5G to the same SSID and rebooting,
but the 2.4G WIFI is still not working.
Only the 5G wifi is working.

[elgab]

23.05.0-rc1
Seems to work OK with 2d uptime. no MT76x2E restart events in log.
I use same SSID at 23dBm for both 2.4GHz and 5GHz + aftermarket 12V 2A power supply.

[dvalter]

Tried running 22.03.5 with 2.4+5 and the same SSID, got an unpleasant experience with clients switching from 2.4 to 5 and back
Then I've got rid of 2.4 completely and it works fine with a few boots of 10-20 days of uptime.

[aaron13223]

From my experience, this is because of the lack of RF Shielding on the routers released later on.... I had the same issue on two of my MI Routers that did not have the RF shield and nothing fixed the issue. I got random restarts and connections drops but mostly it was the error "MAC error detected" and then the wifi would go offline for 15m-1h intermittently unless I manually restarted the router. (I would appreciate if someone can chime in who has the model with the shielding but still has this issue..)

I tried teh snapshot release, 23.X, 22.X. I tried using a higher quality power adapter, made sure no usb peripheral were connected, installed heatsinks and even a fan to rule out overheating issues but only after I made the RF Shields using copper and soldered them on the problem went away. The router has been sooo stable ever since I did that mod. Here's how it turned out:

Copper is so easy to solder on that it was was a breeze, the only time consuming part was bending the copper sheet into the correct size. I also installed thermal pads inside so that it also acts like a heatsink. I used a 0.3mm copper sheet btw.

10/10 would recommend :p

[sirherrbatka]

My router has shielding, but the issue still exists. I managed to improve stability by reducing number of devices connected to the wifi to 2 on 5 ghz and 2 on old wifi.