Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TP-Link Archer A6 v3, stable 21.02.2: Router bricking (5 GHz WiFi broken) #9288

Open
halfbytecode opened this issue Feb 20, 2022 · 23 comments
Open

Comments

@halfbytecode
Copy link

halfbytecode commented Feb 20, 2022

Device: TP-Link Archer A6 v3
OpenWrt version: Stable 21.02.2

Issue: This issue seems to be with 5 GHz WiFi, which is broken and fails to enable, using fresh install as well as after flashing the stable build while preserving the previous configuration (setting with 5 GHz WiFi enabled seems to be the culprit).

If 5 GHz WiFi is enabled and settings force applied from within LuCI, the router is bricked. This resulted in the LEDs going haywire. After soft resetting through failsafe mode, enabling 5 GHz WiFi through LuCI fails and it makes the router reboot with 5 GHz WiFi setting reverted (disabled again). Enabling 2.4 GHz WiFi seems to work.

The most serious is the scenario which I initially faced. Flashing the stable version over the 02/15/2022 snapshot that also seemed to brick my router (with configuration preserved, including 5 GHz WiFi enabled).

The router is also likely to suffer the same fate, if flashing the stable version over a previous stable build with configuration preserved (provided 5 GHz WiFi was enabled).

Other people have also facing the same issue on the forum.

I hope this can be fixed asap, as upgrading with the 5 GHz WiFi enabled may render this router unusable, and perhaps leave it in a bricked state, for more people as well.

I saved both system log and kernel log from within LuCI, if that will help.

System log: https://pastebin.com/SmQtMfum
Kernel log: https://pastebin.com/Zva49nyt

More devices with same chipset MT7621 are also affected by this bug, such as Linksys E5600, as per forum reports, and possibly way more devices.

Source: https://forum.openwrt.org/t/5-ghz-ap-not-working-on-21-02-2-linksys-e5600/120705

@robimarko
Copy link
Contributor

Do you mean downgrade?

Because the snapshot you stated is way newer than 21.02.2

@halfbytecode
Copy link
Author

halfbytecode commented Feb 20, 2022

I meant upgrade as the snapshot is from 3 days before the stable version 21.02.2 was released, with the latter being released on 02/18/2022 whereas the former was from 02/15/2022.

@robimarko
Copy link
Contributor

Well, it doesn't really work like that.
21.01.x is the stable branch and it mostly only gets backported fixes, it's even using a completely different kernel (5.4 vs 5.10) while snapshosts are rolling releases.

So snapshots are always newer and OpenWrt doesnt really support downgrading

@halfbytecode halfbytecode changed the title TP-Link Archer A6 v3, 21.02.2: Router bricking after upgrade (5 GHz WiFi broken) TP-Link Archer A6 v3, stable 21.02.2: Router bricking after upgrade (5 GHz WiFi broken) Feb 20, 2022
@halfbytecode
Copy link
Author

Well, it doesn't really work like that. 21.01.x is the stable branch and it mostly only gets backported fixes, it's even using a completely different kernel (5.4 vs 5.10) while snapshosts are rolling releases.

So snapshots are always newer and OpenWrt doesnt really support downgrading

Okay. After installing the stable version, I had soft reset the router through failsafe mode. That made the router boot again.

At this point, shouldn't the stable version work correctly since I soft reset and removed all my previous configuration?

Enabling 5 GHz WiFi on the stable version crashes the router, and it reboots with 5 GHz WiFi disabled again.

This bug with 5 GHz WiFi is being encountered by multiple people on the forum, including me.

@robimarko
Copy link
Contributor

I agree that it's a bug if 5GHz WLAN doesn't work.
Does it work in snapshots?

@halfbytecode
Copy link
Author

halfbytecode commented Feb 20, 2022

I really have not tried any snapshots after the one from 02/15/2022 where the 5 GHz WiFi did work. However, I faced performance issues and unstability with 5 GHz WiFi on all snapshots I tried, and even the previous stable release v21.02.1. I came to know the mt76 driver is to blame.

Now with 5 GHz WiFi completely broken on stable release v21.02.2, I simply reverted to stock firmware for now.

If it is required to verify the 5 GHz WiFi on the latest snapshot, I will have to reflash OpenWrt, which I can do as required.

@Borromini
Copy link

Please change your topic title to 'downgrade' instead of 'upgrade' - as robimarko pointed out that's what you did. The present title is highly misleading since it implies you upgraded from an older codebase, which you did not. You downgraded from master to 21.02.

@halfbytecode halfbytecode changed the title TP-Link Archer A6 v3, stable 21.02.2: Router bricking after upgrade (5 GHz WiFi broken) TP-Link Archer A6 v3, stable 21.02.2: Router bricking (5 GHz WiFi broken) Feb 21, 2022
@halfbytecode
Copy link
Author

Please change your topic title to 'downgrade' instead of 'upgrade' - as robimarko pointed out that's what you did. The present title is highly misleading since it implies you upgraded from an older codebase, which you did not. You downgraded from master to 21.02.

The issue is not related to upgrade/downgrade, as I was trying to explain in my replies above. I have edited the title and the initial post accordingly, and added more details and findings.

@halfbytecode
Copy link
Author

I agree that it's a bug if 5GHz WLAN doesn't work. Does it work in snapshots?

5 GHz WiFi does work on snapshots. Tried on 02/28/2022 snapshot.

Only the stable release 21.02.2 seems to be affected.

@dmsza
Copy link

dmsza commented Mar 3, 2022

I can confirm this issue on an Archer C6 v3.2 ( same hardware as A6 v3).

Stable release 21.02.2 causes a boot loop when enabling Wi-Fi 5Ghz (fresh install, reset all settings. This does not happen with 21.02.1 nor with snapshot builds.

On the weekend I will try to capture the kernel error with an UART.

@openwrtFan1
Copy link

openwrtFan1 commented Mar 4, 2022

me, too:

stable release 21.02.2, boot loop on archer a6(CA) v3.0
snapshot, as of yesterday, seems to work, at least can connect/use both 2.4/5 ghz; dunno if stable enough, yet.

thanks to all the volunteers of openwrt !!

@dmsza
Copy link

dmsza commented Mar 4, 2022

OK, I just connected an UART to troubleshoot the boot loop issue of stable 21.02.2 with Archer A6 v3 build after enabling the 5Ghz radio. Tested with an Archer C6 v3.2 which has exactly the same hardware (I also backported 21.02.2 to Archer C6 v6 3.2 and did a custom build and the issue is the same).

Below is the portion of the boot log with the error, the full log is at pastebin:

(...)
[   18.736278] br-lan: port 5(wlan1) entered blocking state
[   18.741687] br-lan: port 5(wlan1) entered disabled state
[   18.747797] device wlan1 entered promiscuous mode
[   18.753033] br-lan: port 5(wlan1) entered blocking state
[   18.758501] br-lan: port 5(wlan1) entered forwarding state
[   18.765992] br-lan: port 5(wlan1) entered disabled state
[   18.799528] br-lan: port 6(wlan0) entered blocking state
[   18.805005] br-lan: port 6(wlan0) entered disabled state
[   18.811258] device wlan0 entered promiscuous mode
[   18.816592] br-lan: port 6(wlan0) entered blocking state
[   18.821990] br-lan: port 6(wlan0) entered forwarding state
[   18.829647] IPv6: ADDRCONF(NETDEV_CHANGE): br-lan: link becomes ready
[   18.960970] IPv6: ADDRCONF(NETDEV_CHANGE): wlan0: link becomes ready
[   19.250130] CPU 2 Unable to handle kernel paging request at virtual address 00000000, epc == 86dda8a0, ra == 86ddb5a0
[   19.260868] Oops[#1]:
[   19.263173] CPU: 2 PID: 1543 Comm: hostapd Not tainted 5.4.179 #0
[   19.269252] $ 0   : 00000000 00000001 00000038 00000001
[   19.274463] $ 4   : 00000018 00000008 00000003 00000000
[   19.279670] $ 8   : 00000000 00000064 00000000 00000000
[   19.284878] $12   : 00000000 8681f9e8 00000008 000005ee
[   19.290083] $16   : 00000000 00000001 86816068 86816050
[   19.295289] $20   : 000003e8 00000001 0000145a 00000000
[   19.300506] $24   : 00000002 8680134c
[   19.305728] $28   : 8681e000 8681f7f0 86e17124 86ddb5a0
[   19.310938] Hi    : ccccc547
[   19.313801] Lo    : 66666485
[   19.316692] epc   : 86dda8a0 0x86dda8a0 [mt76_connac_lib@c3ed30b7+0x4e80]
[   19.323451] ra    : 86ddb5a0 0x86ddb5a0 [mt76_connac_lib@c3ed30b7+0x4e80]
[   19.330206] Status: 11007c03 KERNEL EXL IE
[   19.334377] Cause : 40800008 (ExcCode 02)
[   19.338364] BadVA : 00000000
[   19.341231] PrId  : 0001992f (MIPS 1004Kc)
[   19.345304] Modules linked in: pppoe ppp_async iptable_nat xt_state xt_nat xt_conntrack xt_REDIRECT xt_MASQUERADE xt_FLOWOFFLOAD xt_CT pppox ppp_generic nf_nat nf_flow_table_hw nf_flow_table nf_conntrack mt7615e mt7615_common mt7603e mt76_connac_lib mt76 mac80211 ipt_REJECT cfg80211 xt_time xt_tcpudp xt_multiport xt_mark xt_mac xt_limit xt_comment xt_TCPMSS xt_LOG slhc nf_reject_ipv4 nf_log_ipv4 nf_defrag_ipv6 nf_defrag_ipv4 iptable_mangle iptable_filter ip_tables hwmon crc_ccitt compat nf_log_ipv6 nf_log_common ip6table_mangle ip6table_filter ip6_tables ip6t_REJECT x_tables nf_reject_ipv6 leds_gpio gpio_button_hotplug
[   19.399957] Process hostapd (pid: 1543, threadinfo=6195646d, task=3c59292f, tls=77e2dec8)
[   19.408093] Stack : 86815c00 86814c20 86eff230 86814c20 00000000 86e16e60 86815f80 00000001
[   19.416418]         86815f80 86ddb5a0 86815c00 86f49254 00000003 0000145a 00000000 00000000
[   19.424742]         00000000 00000000 00100002 00512a24 04010302 00000100 00000003 0000145a
[   19.433066]         00000000 00000000 00000000 00000000 86815c00 00000000 00200000 00000001
[   19.441389]         00000000 00000000 00000000 00640000 00003802 00000000 0008000f 00000001
[   19.449713]         ...
[   19.452146] Call Trace:
[   19.452184] [<86ddb5a0>] 0x86ddb5a0 [mt76_connac_lib@c3ed30b7+0x4e80]
[   19.461030] [<86f49254>] 0x86f49254 [mac80211@347a072b+0x7fa40]
[   19.466945] [<868014d0>] 0x868014d0 [mt7615_common@09d122ca+0x102a0]
[   19.473296] [<86f27bc8>] 0x86f27bc8 [mac80211@347a072b+0x7fa40]
[   19.479203] [<86eb7ff8>] 0x86eb7ff8 [cfg80211@bb450c4d+0x44270]
[   19.485117] [<86eab3c4>] 0x86eab3c4 [cfg80211@bb450c4d+0x44270]
[   19.491013] [<80152fd4>] 0x80152fd4
[   19.494486] [<8005c158>] 0x8005c158
[   19.498006] [<804a3ccc>] 0x804a3ccc
[   19.501505] [<804a3a9c>] 0x804a3a9c
[   19.504979] [<804a1a14>] 0x804a1a14
[   19.508454] [<804a2258>] 0x804a2258
[   19.511926] [<8017424c>] 0x8017424c
[   19.515404] [<804a2a4c>] 0x804a2a4c
[   19.518886] [<804a1930>] 0x804a1930
[   19.522360] [<8017424c>] 0x8017424c
[   19.525848] [<804a1d0c>] 0x804a1d0c
[   19.529323] [<8017a9e0>] 0x8017a9e0
[   19.532797] [<80425a58>] 0x80425a58
[   19.536276] [<80410b28>] 0x80410b28
[   19.539750] [<80411328>] 0x80411328
[   19.543238] [<80411438>] 0x80411438
[   19.546709] [<802d1330>] 0x802d1330
[   19.550184] [<800707e0>] 0x800707e0
[   19.553664] [<800704b4>] 0x800704b4
[   19.557160] [<80426628>] 0x80426628
[   19.560633] [<804114f0>] 0x804114f0
[   19.564101] [<8040f2ac>] 0x8040f2ac
[   19.567581] [<801d25f4>] 0x801d25f4
[   19.571060] [<80412658>] 0x80412658
[   19.574536] [<8040fb1c>] 0x8040fb1c
[   19.578017] [<80014558>] 0x80014558
[   19.581494]
[   19.582974] Code: 10600002  00801025  34820020 <92030000> 1060ffe4  8fbf0024  1000ffe2  34420040  8c840000
[   19.592685]
[   19.594966] ---[ end trace 05ac765cd135cc09 ]---
[   19.599633] Kernel panic - not syncing: Fatal exception
[   19.604895] Rebooting in 3 seconds..

@dmsza
Copy link

dmsza commented Mar 4, 2022

Adding steps to reproduce:

  1. Install 21.02.2 build without preserving config (clean install)
  2. Logon via LuCI, do basic Wifi config (see below, captured right after step 4 and before device reboots - passwords redacted)
  3. Enable 2.4Ghz radio - OK
  4. Enable 5Ghz radio - NOT OK. Device reboots and enters in a boot loop.
root@OpenWrt:/etc/config# cat wireless

config wifi-device 'radio0'
        option type 'mac80211'
        option path '1e140000.pcie/pci0000:00/0000:00:00.0/0000:01:00.0'
        option channel '1'
        option band '2g'
        option htmode 'HT20'
        option cell_density '0'

config wifi-iface 'default_radio0'
        option device 'radio0'
        option network 'lan'
        option mode 'ap'
        option ssid 'OpenWrt'
        option encryption 'psk2+ccmp'
        option key 'xxxxxxxxx'

config wifi-device 'radio1'
        option type 'mac80211'
        option path '1e140000.pcie/pci0000:00/0000:00:01.0/0000:02:00.0'
        option channel '36'
        option band '5g'
        option htmode 'VHT80'
        option cell_density '0'

config wifi-iface 'default_radio1'
        option device 'radio1'
        option network 'lan'
        option mode 'ap'
        option ssid 'OpenWrt5ghz'
        option encryption 'psk2+ccmp'
        option key 'xxxxxxxxx'

@hnyman
Copy link
Contributor

hnyman commented Mar 5, 2022

Looking at that kernel trace, it looks like something (null pointer access?) that hostapd triggers in mt76 driver.
Cc @nbd168

21.02 branch seems to contain the mt76 driver from December, so it is possibly something that has already been fixed in master.

[   18.960970] IPv6: ADDRCONF(NETDEV_CHANGE): wlan0: link becomes ready
[   19.250130] CPU 2 Unable to handle kernel paging request at virtual address 00000000, epc == 86dda8a0, ra == 86ddb5a0
[   19.260868] Oops[#1]:
[   19.263173] CPU: 2 PID: 1543 Comm: hostapd Not tainted 5.4.179 #0
[   19.269252] $ 0   : 00000000 00000001 00000038 00000001
...
[   19.449713]         ...
[   19.452146] Call Trace:
[   19.452184] [<86ddb5a0>] 0x86ddb5a0 [mt76_connac_lib@c3ed30b7+0x4e80]
[   19.461030] [<86f49254>] 0x86f49254 [mac80211@347a072b+0x7fa40]
[   19.466945] [<868014d0>] 0x868014d0 [mt7615_common@09d122ca+0x102a0]
[   19.473296] [<86f27bc8>] 0x86f27bc8 [mac80211@347a072b+0x7fa40]

@dmsza
Copy link

dmsza commented Mar 5, 2022

Master is ok. The issue is only with 21.02.2.

Perhpas it’s some incompatibility of mt76 with kernel 5.4, since master (for mt7621) switched to kernel 5.10 in December.

@hnyman
Copy link
Contributor

hnyman commented Mar 5, 2022

Master is ok. The issue is only with 21.02.2.

mt76 driver in master has been updated later, but 21.02 has an older version.

@dmsza
Copy link

dmsza commented Mar 5, 2022

I've found the root cause.

It's not the driver. It's a regression of an issue in the build configuration.

The 21.02.2 stable build is including the package kmod-mt7663-firmware-sta and as @nbd explained in another discussion here, it should NOT be included in the builds.

The workaround for now is after installing 21.02.2 stable and before enabling the 5Ghz radio, manually remove package kmod-mt7663-firmware-sta and reboot.

After this the 5Ghz radio can be successfully configured and enabled.

@hnyman
Copy link
Contributor

hnyman commented Mar 5, 2022

probably this commit should be backported from master to 21.02

a1ac872
ramips: remove kmod-mt7663-firmware-sta from device packages

@openwrtFan1
Copy link

openwrtFan1 commented Mar 7, 2022

guest wifi (https://openwrt.org/docs/guide-user/network/wifi/guestwifi/configuration_command_line_interface)
doesn't seem to work with
21.02.2 stable, removing kmod-mt7663-firmware-sta

(works with the snapshot)

hnyman referenced this issue Mar 14, 2022
This firmware should only be used for mobile devices (e.g. laptops), where
AP mode functionality is typically not used. This firmware supports a lot
of power saving offload functionality at the expense of AP mode support.

Signed-off-by: Felix Fietkau <nbd@nbd.name>
@dmsza
Copy link

dmsza commented Mar 19, 2022

All, I've just created a pull request that will both add Archer C6 v3 support to future 21.02 stable builds and also fix this issue.

The pull request basically is a cherry pick of the two commits below:

d22fb7f4fd - ramips: add support for TP-Link Archer C6 v3
a1ac8728f8 - ramips: remove kmod-mt7663-firmware-sta from device packages

The second commit above is the one that fixes the issue reported here in 21.02 branch.

However I need 6 reviews of my pull request so these changes make into 21.02. I would really appreciate if you could review and if OK approve this pull request:

Please review and approve pull request #9502

@hnyman
Copy link
Contributor

hnyman commented Mar 23, 2022

@nbd168 cherry-picked today the culprit commit (remove 7663 sta firmware) to 21.02

c6256a6

@Akczht
Copy link

Akczht commented Aug 29, 2022

how is the performance for Archer a6 v3 now ? I'm looking for a better firmware .

@halfbytecode
Copy link
Author

@Akczht It is quite good for me. I am on a snapshot from March (r19003-43276b60c6), and it is working quite well for me, better than stock firmware.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants