FS#3118 - Failed sysupgrades on TL-WR902ACv1 all 19.07.x stable releases - suspected sysupgrade image issue

[openwrt-bot]

psherman:

• TP-Link TL-WR902AC v1 (ar71xx)
• All sysupgrades images for OpenWrt 19.07.x (.0 - .3) stable release builds

Steps to reproduce:

• Begin with a normal, working installation of stable OpenWrt release (18.06.x or 19.07.x)
• Download the sysupgrade image from the OpenWrt downloads page (https://downloads.openwrt.org/releases/19.07.3/targets/ar71xx/generic/openwrt-19.07.3-ar71xx-generic-tl-wr902ac-v1-squashfs-sysupgrade.bin) - issue present in 19.07.0, 19.07.1, 19.07.2, 19.07.3
• Upload firmware image via LuCI sysupgrade feature
• Uncheck "keep settings" and proceed with flash.
• When device reboots, observe that it seems to be in a boot loop, never completes OpenWrt booting.

Recovery:

• Download factory image from the OpenWrt downloads page (https://downloads.openwrt.org/releases/19.07.3/targets/ar71xx/generic/openwrt-19.07.3-ar71xx-generic-tl-wr902ac-v1-squashfs-factory.bin)
• Configure TFTP server on an available host computer, follow directions for TFTP recovery on the device information page (https://openwrt.org/toh/tp-link/tl-wr902ac_v1)
• Device boots and OpenWrt 19.07.x works properly after TFTP recovery.

Full description of my experience linked to this bug.

When I have some time, I will try the latest snapshot build both via LuCI and via CLI and I'll add comments based on those tests.

[openwrt-bot]

psherman:

Confirmed same issue with snapshot sysupgrade image downloaded May 23. 2020 (OpenWrt SNAPSHOT, r10250-016d1eb)

TFTP recovery is still successful with factory snapshot image (boots normally).
From there, confirmed that sysupgrade CLI upgrade produces same result (failure) as LuCI upgrade: sysupgrade -n /tmp/openwrt-19.07.3-ar71xx-generic-tl-wr902ac-v1-squashfs-sysupgrade.bin

I also timed the boot-loop frequency -- all LEDs flash and ethernetport flaps every 10 seconds.

I am not sure if I'll be able to evaluate the serial console output during the boot-loops in the immediate future (based on available time). If it is necessary, I can try to make a priority for that. Otherwise, hopefully this gives some additional insight.

[openwrt-bot]

psherman:

Another note -- this is ar71xx target platform. This device is not yet available on the ath79 target for 19.07.3 stable release. However, I do see that it is available in the snapshot -- I have not tried it, though.

[openwrt-bot]

adrianschmutzler:

So, just to understand this properly:
The same image works when TFTP-flashed, but doesn't work when used in sysupgrade?

Please try whether the problem persists with ath79 image from master, both for:

ar71xx (working 19.07) -> ath79 (master)
ath79 (master) -> ath79 (master)

[openwrt-bot]

psherman:

Sorry if it was at all unclear...

The TL-WR902AC has two images available in the 19.07.x stable build directories (ar71xx) -- "factory" which is used for flashing from the original TP-Link firmware and/or tftp recovery, and a "sysupgrade"image which is for upgrading from a previous OpenWrt version. I believe that the difference is supposed to be padding or some header information since the functional part of the image should theoretically be the same.

For example, from the current [[https://downloads.openwrt.org/releases/19.07.3/targets/ar71xx/generic/|External Link19.07.3 stable release]]:
tl-wr902ac-v1-squashfs-factory.bin [sha hash omitted] 4374.0 KB Sun May 17 04:39:47 2020
tl-wr902ac-v1-squashfs-sysupgrade.bin [sha hash omitted] 4352.3 KB Sun May 17 04:39:47 2020

All versions referenced below are for stable builds:

• //legacy// flashing using 'sysupgrade' image from any 18.06.x to any 18.06.x worked without issue (I never had a failed flash).

• flashing using the 'sysupgrade' image (from any 18.06.x or 19.07.x) to any 19.07.x fails every time resulting in a boot loop.

• tftp recovery, using the 'factory' image for all 19.07.x stable builds has been successful each time I have needed to recover (19.07.0, 19.07.1, 19.07.2, 19.07.3)

• I have also tested the ar71xx target snapshot referenced earlier in this thread with the same results.

I will test with ath79 as soon as I can and reply back to this thread with the results.

[openwrt-bot]

adrianschmutzler:

ar71xx doesn't build snapshots for master, the code there is source-only and the images are from June 2019 (so, effectively older than what's offered for 19.07).

Please try the ath79 from master, which should be up-to-date.

[openwrt-bot]

psherman:

Sure thing. Thanks for the clarification. I'll report back when I have results.

[openwrt-bot]

adrianschmutzler:

Just found something. Try to build the referenced branch or pick the most recent commit to a local openwrt-19.07 branch of yours:
https://git.openwrt.org/?p=openwrt/staging/adrian.git;a=shortlog;h=refs/heads/staging-19.07

This could fix the sysupgrade problem for ar71xx.

I'd still expect ath79 in master to work out-of-the-box.

[openwrt-bot]

psherman:

Test ath79 (snapshot) sysupgrade image (for all tests: keep settings disabled, or -n argument):

19.07.3 stable release (ar71xx) > snapshot (ath79) 'sysupgrade' image = successful flash!
snapshot (ath79) > snapshot (ath79) 'sysupgrade' image = successful flash!
snapshot (ath79) > 19.07.3 stable release (ar71xx) 'sysupgrade' image = fail (as expected; tftp recovery necessary)

Interestingly, ar71xx > ath79 did not throw any flags about supported image type (ath79 > ar71xx did warn about the supported image; used sysupgrade -F -n).

I failed to make note of the snapshot version, but it was from the following:
tplink_tl-wr902ac-v1-squashfs-factory.bin [hash omitted] 4443.1 KB Thu May 28 20:45:18 2020

[openwrt-bot]

psherman:

Adrian - I don't know that I'll able to get to test yourr patch in the near future (and I may have a little learning to do if I want to build with that patch).

Will the ath79 target for this device be built for 19.07.4? Or will we need to wait for 20.x to see the new target for this the WR902AC?

If the ar71xx builds will be provided for at least the rest of the 19.07.x series, it is probably worth the effort to test the patch... do you know when the ar71xx target will be removed for this device?

[openwrt-bot]

adrianschmutzler:

Hi, I've built images for ar71xx@19.07 for you and uploaded them here:
https://www.adrianschmutzler.net/upload/wr902ac.zip

These are just containing the default packages, no GUI, so mostly just like the snapshot from master. It would be quite helpful if you could test these, because if I had the correct suspicion we would be able to fix this for 19.07 users.

Please test flashing:
19.07.3 "broken" factory -> "my" sysupgrade
"my" factory -> "my" sysupgrade

if you have additional time, you may also try 18.06 factory -> "my" sysupgrade

Concerning ar71xx vs. ath79:
19.07 will be the last release containing ar71xx. It won't be included in 20.xx, and it's just left in master as source-only to ease development/transfer to ath79. Everything is supposed to be migrated to ath79, and the subset of devices that has been available then is available in 19.07 already. However, device support is typically not backported, so devices added after the branch (i.e. WR902AC) won't be backported to 19.07, particularly since there is a ar71xx variant available for that same device.
So, I'd be happy if we could fix the ar71xx support in 19.07 with this patch.

[openwrt-bot]

adrianschmutzler:

Interestingly, ar71xx > ath79 did not throw any flags about supported image type (ath79 > ar71xx did warn about the supported image; used sysupgrade -F -n).

That's expected, we "support" upgrading, but not downgrading. Despite, downgrading would require to update the "old" stable branch in order to know what's changed in the "new" master branch ...

[openwrt-bot]

psherman:

Thanks Adrian. I tried to build my own images yesterday, but was not successful in integrating the patch. So it was good that you were able to provide images for me to test.

Looks like it is all working exactly as expected! I ran what I think is a comprehensive set of tests -- here is my test log:

19.07.3 (stable release) 'factory' --> 19.07.3 (stable release) 'sysupgrade' image = fail (expected)
failed (bootlooping) --> tftp recovery with Adrian's 'factory' image = success
Adrian's 'factory' image --> Adrian's 'sysupgrade' image = success
Adrian's 'sysupgrade' image --> 18.06.8 (stable release) 'sysupgrade' = success
18.06.8 (stable release) 'sysupgrade' --> Adrian's 'sysupgrade' image = success

force tftp --> 18.06.8 (stable release) 'factory' image = success
18.06.8 'factory' image --> Adrian's 'sysupgrade' image = success
Adrian's 'sysupgrade' image --> Adrian's 'sysupgrade' image = success

force tftp --> 19.07.3 (stable release) 'factory' image = success
19.07.3 (stable release) 'factory' image --> Adrian's 'sysupgrade' image = success

[openwrt-bot]

psherman:

Thanks for fixing this bug, Adrian. I appreciate your help and I'm glad you found the solution so quickly!

[openwrt-bot]

None:

Perfectly executed and solved !!