Update the Login Process and the Script Update Process for MerlinAU.sh by ExtremeFiretop · Pull Request #36 · ExtremeFiretop/MerlinAutoUpdate-Router

ExtremeFiretop · 2023-12-17T19:19:54Z

Started to address known issues.

Script logins into the WebUI before the final prompt to continue.
(This has been resolved by moving the login sequence behind/after the final prompt)
Script update process does not relaunch the updated script.
(This was resolved by having the script re-execute itself into memory right after update.)

ExtremeFiretop · 2023-12-17T19:25:56Z

Still remaining:

On the AX88U Pro, it never completed the flash from script on the second run, the first run failed because I was logged into the web UI, the second run froze at "Flashing, please wait for reboot" and didn't reboot or do anything for 5+ minutes.

Finally, I logged into the web UI, rebooted the router, and update had not completed. I'll be digging through the logs myself to see what happened.

ExtremeFiretop · 2023-12-17T19:26:55Z

My only guess so far on the AX88U Pro is that it's file extension is different with: ".pkgtb" instead of a ".w" once extracted from the .zip

ExtremeFiretop · 2023-12-17T20:13:20Z

Exactly what I thought the problem was.

At lines ~1798 we specify the file type to find: "pure_file="$(ls | grep -iE '_pureubi.w|_ubi.w' | grep -iv 'rog')""

I think it's best we use a looser matching method which can work for all file types.

ExtremeFiretop · 2023-12-17T20:23:10Z

Modified the grep match to include the new .pkgtb file type.

ExtremeFiretop · 2023-12-17T20:45:25Z

So that change correctly flagged and selected the new .pkgtb file as found in the below logs:

RT-AX88U_Pro_FW_Update_2023-12-17_15_32_02.log
[syslog.txt]

However, it still "froze" at the flashing step. In the logs at around 15:33

**Dec 17 15:33:34 [MerlinAU.sh] 13255: Flashing ^[[1;32mRT-AX88U_PRO_3004_388.5_0_nand_squashfs.pkgtb^[[0m... ^[[1;31mPlease Wait for Reboot.^[[0m**
**Dec 17 15:33:34 rc_service: httpd 2287:notify_rc stop_upgrade**
Dec 17 15:33:34 rc_service: waitting "restart_leds" via  ...
Dec 17 15:33:43 rc_service: skip the event: restart_leds.
Dec 17 15:33:45 rc_service: service 14169:notify_rc restart_leds
Dec 17 15:33:45 rc_service: waitting "restart_leds" via  ...
Dec 17 15:33:47 kernel: DROP IN=ppp0 OUT= MAC= SRC=65.34.19.85 DST=70.24.148.241 LEN=44 TOS=0x00 PREC=0x00 TTL=47 ID=21735 PROTO=TCP SPT=11611 DPT=23 SEQ=1176016113 ACK=0 WINDOW=63835 RES=0x00 SYN URGP=0 OPT (020405B4) MARK=0x8000000 
Dec 17 15:33:47 kernel: DROP IN=ppp0 OUT= MAC= SRC=92.63.197.224 DST=70.24.148.241 LEN=40 TOS=0x00 PREC=0x00 TTL=240 ID=22318 PROTO=TCP SPT=53252 DPT=18630 SEQ=1961265945 ACK=0 WINDOW=1024 RES=0x00 SYN URGP=0 MARK=0x8000000 
**Dec 17 15:33:48 rc_service: skip the event: stop_upgrade.**

MerlinAU.sh

ExtremeFiretop · 2023-12-17T20:51:48Z

MerlinAU.sh

    # DEBUG: Print the LAN IP to ensure it's being set correctly
    printf "\n**DEBUG**: Router Web URL is: ${routerURLstr}\n"

+    printf "${GRNct}**IMPORTANT**:${NOct}\nThe firmware flash is about to start.\n"


Moved this up from below. (where the routers login happens.)

ExtremeFiretop · 2023-12-17T20:52:22Z

MerlinAU.sh

-         echo -e "${CYANct}Download successful!${NOct}"
         echo -e "$(date) - $SCRIPT_NAME - Successfully downloaded $SCRIPT_NAME v$DLRepoVersion"
+	 echo -e "${CYANct}Update successful! Restarting script...${NOct}"
+	 exec "${ScriptsDirPath}/${SCRIPT_NAME}.sh"  # Re-execute the updated script


This exec will relaunch the new version of the script at the end.

ExtremeFiretop · 2023-12-17T21:02:35Z

I couldn't reboot the router in the WebUI. Had to reboot it via ssh typing "reboot".
Pending it restart and I'll continue to troubleshoot this newer model of the AX88U Pro

ExtremeFiretop · 2023-12-17T21:04:42Z

WEIRD. Post reboot it was updated lol. It just didn't reboot itself? WEIRD.

ExtremeFiretop · 2023-12-17T21:12:49Z

Extended the sleep from 60 seconds to 3 minutes to be "sure" the update is done.
Then throwing an extra "reboot" command in at line ~1888 for certain routers which need a manual reboot at the end.
(Like my AX88U Pro)

ExtremeFiretop · 2023-12-17T21:15:08Z

MerlinAU.sh

        -F "file=@${firmware_file}" \
        --cookie /tmp/cookie.txt > /tmp/upload_response.txt 2>&1 &
-        sleep 60
+        sleep 180


Extended the sleep timer from 60 to 180 (3 minutes, the default amount in the webUI when doing an upgrade) to be sure the update should be completed before we throw a reboot command.

ExtremeFiretop · 2023-12-17T21:15:24Z

MerlinAU.sh

        --cookie /tmp/cookie.txt > /tmp/upload_response.txt 2>&1 &
-        sleep 60
+        sleep 180
+        /sbin/service reboot


Adding a reboot for some routers after 3 minutes of waiting if it didn't reboot itself.

I just had an idea to improve this crucial point in the process. Let me code it and run some testing before I put it in the script.

ExtremeFiretop · 2023-12-17T21:41:54Z

MerlinAU.sh

 	# Detect ROG and pure firmware files
 	rog_file="$(ls | grep -i '_rog_')"
-	pure_file="$(ls | grep -iE '(_pureubi\.w|_ubi\.w|.*_nand_squashfs\.pkgtb)$' | grep -iv 'rog')"
+	pure_file="$(ls | grep -iE '(\.w|\.pkgtb|\.trx)$' | grep -iv 'rog')"


Only match by file extensions, and remove any found files with "rog" in the name.

ExtremeFiretop · 2023-12-18T00:24:27Z

I can confirm I got a successful upgrade on the AX88U Pro using these fixes.

ExtremeFiretop · 2023-12-18T00:49:10Z

I also unblocked these routers:

RT-AC68U
RT-AC66U_B1
RT-AC1900

Here: 0a8d58e

As I never got a response from Yota in the forums, and I believe the changes in logic we've made for memory and requiring a USB on these models should technically allow for it to work. Until someone tests and provides logs that the latest dev build doesn't work, I will claim they are supported.

ExtremeFiretop · 2023-12-18T01:47:40Z

However, that being said, I just noticed that the naming scheme for those routers firmware is different:

https://sourceforge.net/projects/asuswrt-merlin/files/RT-AC68U/Release/

Makes me think maybe we would need to account for this since the actual flash right now uses the dotted version of the 3004 versioning right now for example.

ExtremeFiretop · 2023-12-18T01:52:08Z

nvm.

We say that the RT-AC86U works and it uses the same naming scheme so we already account for it:
https://sourceforge.net/projects/asuswrt-merlin/files/RT-AC86U/Release/

That is your model right? How does the flash work when it's using the dotted version of the firmware?

ExtremeFiretop · 2023-12-18T02:12:30Z

Nvm. I got my answer, it dots the first octs of the release version no matter what it is.
In this case: 386.12.4 becomes: 3.8.6 for the flash command.

Martinski4GitHub · 2023-12-18T05:52:29Z

Exactly what I thought the problem was.

At lines ~1798 we specify the file type to find: "pure_file="$(ls | grep -iE '_pureubi.w|_ubi.w' | grep -iv 'rog')""

I think it's best we use a looser matching method which can work for all file types.

Excellent catch!!

Martinski4GitHub · 2023-12-18T06:00:44Z

nvm.

We say that the RT-AC86U works and it uses the same naming scheme so we already account for it: https://sourceforge.net/projects/asuswrt-merlin/files/RT-AC86U/Release/

That is your model right? How does the flash work when it's using the dotted version of the firmware?

Yes, that's correct. The release versioning schemes & eventual comparison are handled via the functions GetCurrentFWInstalledShortVersion() & FWVersionStrToNum() which take into account the "old" and the "new" format.

Martinski4GitHub · 2023-12-18T06:03:00Z

I can confirm I got a successful upgrade on the AX88U Pro using these fixes.

Excellent! It looks like you had a very productive day chasing the issues with AX88U Pro router. Good job buddy!!

ExtremeFiretop · 2023-12-18T06:24:16Z

I can confirm I got a successful upgrade on the AX88U Pro using these fixes.

Excellent! It looks like you had a very productive day chasing the issues with AX88U Pro router. God job buddy!!

I had some free time today, and just wanted to add another model tested on the list, I realized my side gig had an AX88U Pro which I had access too, so I tested it expecting it to just work.

When it didn't, it sent me down a rabbit hole of tweaks and fixes until it worked.

I know this was lots to read, it was mostly taking notes for myself and my thought process while troubleshooting. Lol.

The big take aways today was:

I adjusted the script so the login happens after the final prompt instead of before
I fixed the matching with grep so it should work with all the file types. (And extended the sleep and added a reboot for safety in case it doesn't do it itself)
And adjusted the update function to relaunch the script once updated.

As usual if you notice anything I did was wrong or makes a bug please let me know. Your the master!

ExtremeFiretop · 2023-12-18T06:27:08Z

@Martinski4GitHub

Are you also confirming it's worked for you using the RT-AC86U? Is that your router too? (My friend has it)

My friend is still using my old desktop script for now ... but I updated the desktop script to use this curl method so he's been happy with that until I convince him to migrate to this.

Martinski4GitHub · 2023-12-18T06:42:45Z

I can confirm I got a successful upgrade on the AX88U Pro using these fixes.

Excellent! It looks like you had a very productive day chasing the issues with AX88U Pro router. God job buddy!!

I had some free time today, and just wanted to add another model tested on the list, I realized my side gig had an AX88U Pro which I had access too, so I tested it expecting it to just work.

When it didn't, it sent me down a rabbit hole of tweaks and fixes until it worked.

That was some very nice sleuthing you did today, and it was a very serendipitous discovery with the AX88U Pro router, just in time before promoting the latest "Dev" version to the "Main" branch.

I know this was lots to read, it was mostly taking notes for myself and my thought process while troubleshooting. Lol.

The big take aways today was:
1. I adjusted the script so the login happens after the final prompt instead of before

2. I fixed the matching with grep so it should work with all the file types. (And extended the sleep and added a reboot for safety in case it doesn't do it itself)

3. And adjusted the update function to relaunch the script once updated.

I'm starting to review the latest changes. It all looks good so far.

As usual if you notice anything I did was wrong or makes a bug please let me know. Your the master!

Will do. But I have not yet achieved the "master" status; it takes patience & perseverance, Grasshopper!

P.S. I hope you get the reference to the TV show :>).

Martinski4GitHub · 2023-12-18T06:49:05Z

@Martinski4GitHub

Are you also confirming it's worked for you using the RT-AC86U? Is that your router too? (My friend has it)

Yes, I've run the full F/W update process only once, and it did work without any issues. I realize that one data sample is not too convincing, but I also run a "test/debug" version that I have which includes some carefully placed "printf" statements to stdout to display what's happening as it executes until the point just before the actual "curl" command that does the final update.

Update MerlinAU.sh

796ff4c

ExtremeFiretop requested a review from Martinski4GitHub December 17, 2023 19:19

ExtremeFiretop assigned ExtremeFiretop and Martinski4GitHub Dec 17, 2023

ExtremeFiretop added the bug Something isn't working label Dec 17, 2023

ExtremeFiretop added 2 commits December 17, 2023 14:21

Update MerlinAU.sh

a5189ac

Update MerlinAU.sh

923fc41

ExtremeFiretop mentioned this pull request Dec 17, 2023

Update MerlinAU.sh #35

Merged

Update version.txt

5104de0

ExtremeFiretop added 2 commits December 17, 2023 15:19

Update MerlinAU.sh

7c977dc

Update MerlinAU.sh

e99cde0

ExtremeFiretop commented Dec 17, 2023

View reviewed changes

ExtremeFiretop added 2 commits December 17, 2023 16:10

Update MerlinAU.sh

2bee87d

Update MerlinAU.sh

a630bab

ExtremeFiretop added the enhancement New feature or request label Dec 17, 2023

Update MerlinAU.sh

e204088

ExtremeFiretop commented Dec 17, 2023

View reviewed changes

ExtremeFiretop added 2 commits December 17, 2023 16:18

Update README.md

8656353

Update MerlinAU.sh

0285646

ExtremeFiretop commented Dec 17, 2023

View reviewed changes

Update MerlinAU.sh

f46c553

ExtremeFiretop merged commit 1d68e25 into dev Dec 18, 2023

ExtremeFiretop deleted the ExtremeFiretop-patch-1 branch December 18, 2023 00:27

Conversation

ExtremeFiretop commented Dec 17, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ExtremeFiretop commented Dec 17, 2023

Uh oh!

ExtremeFiretop commented Dec 17, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ExtremeFiretop commented Dec 17, 2023

Uh oh!

ExtremeFiretop commented Dec 17, 2023

Uh oh!

ExtremeFiretop commented Dec 17, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

ExtremeFiretop Dec 17, 2023

Choose a reason for hiding this comment

Uh oh!

ExtremeFiretop Dec 17, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ExtremeFiretop commented Dec 17, 2023

Uh oh!

ExtremeFiretop commented Dec 17, 2023

Uh oh!

ExtremeFiretop commented Dec 17, 2023

Uh oh!

ExtremeFiretop Dec 17, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ExtremeFiretop Dec 17, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Martinski4GitHub Dec 18, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ExtremeFiretop Dec 17, 2023

Choose a reason for hiding this comment

Uh oh!

ExtremeFiretop commented Dec 18, 2023

Uh oh!

ExtremeFiretop commented Dec 18, 2023

Uh oh!

ExtremeFiretop commented Dec 18, 2023

Uh oh!

ExtremeFiretop commented Dec 18, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ExtremeFiretop commented Dec 18, 2023

Uh oh!

Martinski4GitHub commented Dec 18, 2023

Uh oh!

Martinski4GitHub commented Dec 18, 2023

Uh oh!

Martinski4GitHub commented Dec 18, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ExtremeFiretop commented Dec 18, 2023

Uh oh!

ExtremeFiretop commented Dec 18, 2023

Uh oh!

Martinski4GitHub commented Dec 18, 2023

Uh oh!

Martinski4GitHub commented Dec 18, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ExtremeFiretop commented Dec 17, 2023 •

edited

Loading

ExtremeFiretop commented Dec 17, 2023 •

edited

Loading

ExtremeFiretop commented Dec 17, 2023 •

edited

Loading

ExtremeFiretop Dec 17, 2023 •

edited

Loading

ExtremeFiretop Dec 17, 2023 •

edited

Loading

ExtremeFiretop Dec 17, 2023 •

edited

Loading

Martinski4GitHub Dec 18, 2023 •

edited

Loading

ExtremeFiretop commented Dec 18, 2023 •

edited

Loading

Martinski4GitHub commented Dec 18, 2023 •

edited

Loading