Skip to content

Update Memory Management#167

Merged
ExtremeFiretop merged 16 commits intodevfrom
ExtremeFiretop-Memory_Management
Mar 24, 2024
Merged

Update Memory Management#167
ExtremeFiretop merged 16 commits intodevfrom
ExtremeFiretop-Memory_Management

Conversation

@ExtremeFiretop
Copy link
Owner

Update Memory Management

Update Memory Management
@ExtremeFiretop
Copy link
Owner Author

Results:

Required RAM: 227272 KB - RAM Free: 388024 KB - RAM Available: 526392 KB
-----------------------------------------------------------
Archive:  /tmp/mnt/USB2/MerlinAU.d/GT-AXE11000_firmware/GT-AXE11000_firmware.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
 85065748  02-26-2024 23:01   GT-AXE11000_3004_388.6_2_pureubi.w
     9645  02-26-2024 21:20   README-merlin.txt
    94437  02-26-2024 22:21   Changelog-386.txt
    99381  02-26-2024 21:20   Changelog-NG.txt
      206  02-26-2024 23:10   sha256sum.sha256
 87687188  02-26-2024 23:10   GT-AXE11000_3004_388.6_2_rog_pureubi.w
 --------                     -------
172956605                     6 files
-----------------------------------------------------------
Archive:  /tmp/mnt/USB2/MerlinAU.d/GT-AXE11000_firmware/GT-AXE11000_firmware.zip
  inflating: GT-AXE11000_3004_388.6_2_pureubi.w
  inflating: Changelog-386.txt
  inflating: Changelog-NG.txt
  inflating: sha256sum.sha256
  inflating: GT-AXE11000_3004_388.6_2_rog_pureubi.w
-----------------------------------------------------------
Adjusted required RAM by subtracting sizes of .w and .pkgtb files: 168704 KB. New required RAM: 58568 KB
Required RAM: 58568 KB - RAM Free: 218960 KB - RAM Available: 357704 KB

@Yaff1e
Copy link

Yaff1e commented Mar 19, 2024

Good news. I've just tried with v1..0.9 and to my untrained eye it looks as if it worked perfectly on my AX86S with low free RAM.

2024-03-19 23:21:36 Required RAM: 112207 KB - RAM Free: 121512 KB - RAM Available: 123640 KB
2024-03-19 23:21:36 Backup Started (by BACKUPMON)
2024-03-19 23:22:45 Backup Finished
2024-03-19 23:22:45 Backup Completed Successfully
2024-03-19 23:22:45 Latest release version is 3004.388.6.2.
2024-03-19 23:22:45 Downloading https://sourceforge.net/projects/asuswrt-merlin/files/RT-AX86U/Release/RT-AX86U_3004_388.6_2.zip
2024-03-19 23:22:50 Required RAM: 112207 KB - RAM Free: 31444 KB - RAM Available: 120384 KB
2024-03-19 23:22:50 -----------------------------------------------------------
2024-03-19 23:22:50 Archive:  /tmp/mnt/Kingston/MerlinAU.d/RT-AX86U_firmware/RT-AX86U_firmware.zip
2024-03-19 23:22:50   Length      Date    Time    Name
2024-03-19 23:22:50 ---------  ---------- -----   ----
2024-03-19 23:22:50  85196820  02-26-2024 23:01   RT-AX86U_3004_388.6_2_pureubi.w
2024-03-19 23:22:50      9645  02-26-2024 21:20   README-merlin.txt
2024-03-19 23:22:50     94437  02-26-2024 22:21   Changelog-386.txt
2024-03-19 23:22:50     99381  02-26-2024 21:20   Changelog-NG.txt
2024-03-19 23:22:50        98  02-26-2024 23:02   sha256sum.sha256
2024-03-19 23:22:50  --------                     -------
2024-03-19 23:22:50  85400381                     5 files
2024-03-19 23:22:50 -----------------------------------------------------------
2024-03-19 23:23:31 Archive:  /tmp/mnt/Kingston/MerlinAU.d/RT-AX86U_firmware/RT-AX86U_firmware.zip
2024-03-19 23:23:31   inflating: RT-AX86U_3004_388.6_2_pureubi.w
2024-03-19 23:23:31   inflating: Changelog-386.txt
2024-03-19 23:23:31   inflating: Changelog-NG.txt
2024-03-19 23:23:31   inflating: sha256sum.sha256
2024-03-19 23:23:31 -----------------------------------------------------------
2024-03-19 23:23:33 Total size of files: 85196820 bytes
2024-03-19 23:23:33 Adjusted required RAM by subtracting sizes of .w and .pkgtb files: 83200 KB. New required RAM: 29007 KB
2024-03-19 23:23:33 Required RAM: 29007 KB - RAM Free: 30836 KB - RAM Available: 39884 KB
2024-03-19 23:23:33 No high-risk phrases found in the change-log.
2024-03-19 23:23:33 Required RAM: 29007 KB - RAM Free: 30604 KB - RAM Available: 39556 KB
2024-03-19 23:23:33 No ROG Build detected. Skipping.
2024-03-19 23:23:33 Required RAM: 29007 KB - RAM Free: 29896 KB - RAM Available: 39324 KB
2024-03-19 23:23:34 Router Web URL is: https://192.168.5.2:8442
2024-03-19 23:24:41 The email notification was sent successfully [START_FW_UPDATE_STATUS].
2024-03-19 23:24:41 Stopping Entware services...
2024-03-19 23:24:57 Post-update email notification hook was added successfully to '/jffs/scripts/services-start' script.
2024-03-19 23:24:57 Flashing RT-AX86U_3004_388.6_2_pureubi.w... Please wait for reboot in about 4 minutes or less.

It's now showing as the most recent version and everything seems to be okay up to now.

I'll forward the logs on the existing SNB conversation.

@ExtremeFiretop
Copy link
Owner Author

Good news. I've just tried with v1..0.9 and to my untrained eye it looks as if it worked perfectly on my AX86S with low free RAM.

2024-03-19 23:21:36 Required RAM: 112207 KB - RAM Free: 121512 KB - RAM Available: 123640 KB
2024-03-19 23:21:36 Backup Started (by BACKUPMON)
2024-03-19 23:22:45 Backup Finished
2024-03-19 23:22:45 Backup Completed Successfully
2024-03-19 23:22:45 Latest release version is 3004.388.6.2.
2024-03-19 23:22:45 Downloading https://sourceforge.net/projects/asuswrt-merlin/files/RT-AX86U/Release/RT-AX86U_3004_388.6_2.zip
2024-03-19 23:22:50 Required RAM: 112207 KB - RAM Free: 31444 KB - RAM Available: 120384 KB
2024-03-19 23:22:50 -----------------------------------------------------------
2024-03-19 23:22:50 Archive:  /tmp/mnt/Kingston/MerlinAU.d/RT-AX86U_firmware/RT-AX86U_firmware.zip
2024-03-19 23:22:50   Length      Date    Time    Name
2024-03-19 23:22:50 ---------  ---------- -----   ----
2024-03-19 23:22:50  85196820  02-26-2024 23:01   RT-AX86U_3004_388.6_2_pureubi.w
2024-03-19 23:22:50      9645  02-26-2024 21:20   README-merlin.txt
2024-03-19 23:22:50     94437  02-26-2024 22:21   Changelog-386.txt
2024-03-19 23:22:50     99381  02-26-2024 21:20   Changelog-NG.txt
2024-03-19 23:22:50        98  02-26-2024 23:02   sha256sum.sha256
2024-03-19 23:22:50  --------                     -------
2024-03-19 23:22:50  85400381                     5 files
2024-03-19 23:22:50 -----------------------------------------------------------
2024-03-19 23:23:31 Archive:  /tmp/mnt/Kingston/MerlinAU.d/RT-AX86U_firmware/RT-AX86U_firmware.zip
2024-03-19 23:23:31   inflating: RT-AX86U_3004_388.6_2_pureubi.w
2024-03-19 23:23:31   inflating: Changelog-386.txt
2024-03-19 23:23:31   inflating: Changelog-NG.txt
2024-03-19 23:23:31   inflating: sha256sum.sha256
2024-03-19 23:23:31 -----------------------------------------------------------
2024-03-19 23:23:33 Total size of files: 85196820 bytes
2024-03-19 23:23:33 Adjusted required RAM by subtracting sizes of .w and .pkgtb files: 83200 KB. New required RAM: 29007 KB
2024-03-19 23:23:33 Required RAM: 29007 KB - RAM Free: 30836 KB - RAM Available: 39884 KB
2024-03-19 23:23:33 No high-risk phrases found in the change-log.
2024-03-19 23:23:33 Required RAM: 29007 KB - RAM Free: 30604 KB - RAM Available: 39556 KB
2024-03-19 23:23:33 No ROG Build detected. Skipping.
2024-03-19 23:23:33 Required RAM: 29007 KB - RAM Free: 29896 KB - RAM Available: 39324 KB
2024-03-19 23:23:34 Router Web URL is: https://192.168.5.2:8442
2024-03-19 23:24:41 The email notification was sent successfully [START_FW_UPDATE_STATUS].
2024-03-19 23:24:41 Stopping Entware services...
2024-03-19 23:24:57 Post-update email notification hook was added successfully to '/jffs/scripts/services-start' script.
2024-03-19 23:24:57 Flashing RT-AX86U_3004_388.6_2_pureubi.w... Please wait for reboot in about 4 minutes or less.

It's now showing as the most recent version and everything seems to be okay up to now.

I'll forward the logs on the existing SNB conversation.

Thank you for testing and reporting your results @Yaff1e

@ExtremeFiretop
Copy link
Owner Author

@Martinski4GitHub

Added the minimum cushion we discussed. Will test it sometime this weekend.

@ExtremeFiretop
Copy link
Owner Author

ExtremeFiretop commented Mar 20, 2024

Lightly upped the cushion, just to be absolutely sure the steps following don't trigger swapping.

@Martinski4GitHub
Copy link
Collaborator

@Martinski4GitHub

Added the minimum cushion we discussed. Will test it sometime this weekend.

Sounds good. Looking forward to hearing about your test results.

@Martinski4GitHub
Copy link
Collaborator

Lightly upped the cushion, just to be absolutely sure the steps following don't trigger swapping.

OK, that's good. At this point, "better safe than sorry" would be the best approach. I really don't want our script to be blamed if a router is bricked after trying to flash the F/W image while in a "low memory" condition. As we get more data points, and hopefully a few users are willing to test the new code and report results, we can adjust accordingly.

@ExtremeFiretop
Copy link
Owner Author

Lightly upped the cushion, just to be absolutely sure the steps following don't trigger swapping.

OK, that's good. At this point, "better safe than sorry" would be the best approach. I really don't want our script to be blamed if a router is bricked after trying to flash the F/W image while in a "low memory" condition. As we get more data points, and hopefully a few users are willing to test the new code and report results, we can adjust accordingly.

For this user above, the 35MB limit would still allow his update to start, it would just try to reclaim memory if it fell below 35MB going forwards. I think this may be the safe way forwards while letting as many routers through the gate as possible that can safely update.

@Martinski4GitHub
Copy link
Collaborator

Martinski4GitHub commented Mar 20, 2024

For this user above, the 35MB limit would still allow his update to start, it would just try to reclaim memory if it fell below 35MB going forwards. I think this may be the safe way forwards while letting as many routers through the gate as possible that can safely update.

Fully agreed. I still wonder what's causing his router to get so low in "Available RAM" after the ZIP is downloaded and the image is stored in the HOME folder. During all my testing with the RT-AC86U, it has never reached the point where it was below the original "Required RAM" amount - not a single time. And I've been testing for a few months now, using different F/W versions. Most of the time, I have stopped short of the actual F/W flash, but it's already after the point where it has checked for the minimum required and has passed so I get the prompt asking me to type Y to continue to flash.

I feel that knowing what's causing his router to get in such a "low memory" condition may give us some clues about what to do to either avoid it or take further steps to get more of the RAM back. Just a thought.

@ExtremeFiretop
Copy link
Owner Author

ExtremeFiretop commented Mar 20, 2024

@Martinski4GitHub

Completed the BACKUPMON Toggle for the Advanced Options menu.

So far this PR includes:

  1. Improved Memory Management.
    With a cushion upped to 40MB from 35MB.
    I've decided safer is better than sorry as you mentioned yesterday, and based on the data we have today, this user above would of worked in both his attempts according to his logs if the cushion was at 40MB, first run would of passed no problem from the logs in the forums, second run according to his logs here would of triggered the cleanup. (But we know that frees more than 1MB of RAM so he would of flashed successfully). Also the Kernel will always be more responsive than our script. The kernel would near instantly trigger swapping while our script only checks at standard intervals, so I rather give the script as much headroom as possible.

  2. Removed the unnecessary Migration Function.
    As we discussed, this next release will drop the migration function, it's been around for months so we should be safe to remove and free some space in the script for logic that will actually be used. I always knew this function I built would be temporary.

  3. Added the Toggle option in the Advanced Options for BACKUPMON.
    As we discussed, this next release will give the user control over the BACKUPMON backups. Toggle is placed in the Advanced Options Menu and is hidden if BACKUPMON is not installed. Default Options is still "Enabled". This is incase the user already has daily backups configured and simply wants MerlinAU to handle Auto-Updates.

@ExtremeFiretop
Copy link
Owner Author

ExtremeFiretop commented Mar 20, 2024

I know I suck, I upped the cushion from 32, to 33 to 35 to 40 lol. Can you tell im indecisive?

No but I'm happy with 40. I thought about it overnight in my dreams. That gives our script double the steps the kernel gives itself. That should be enough headroom for the script to trigger a cleanup if it falls below between the checks.

And also still allow this user with this data we have today to successfully flash. Looked at both his logs, first would of been flawless, second would of needed cleanup, which would given him over 20MB headroom to continue and he was only 1MB below. I personally would of been happier to see it try to cleanup before the flash in his last successful run, this change would of done that. We can re-assess if it becomes a problem again, but considering how close we are to the limits now. It's unlikely we see this issue again, and if we do I'd probably only consider lowering it down to 35 in the future due to safety concerns .

@ExtremeFiretop
Copy link
Owner Author

Touched up some of the logging so we can get logging slightly earlier in the run, can get the version number in the log files.

@Martinski4GitHub
Copy link
Collaborator

@Martinski4GitHub

Completed the BACKUPMON Toggle for the Advanced Options menu.

So far this PR includes:

1. Improved Memory Management.
   **With a cushion upped to 40MB from 35MB.**
   I've decided safer is better than sorry as you mentioned yesterday, and based on the data we have today, this user above would of worked in both his attempts according to his logs if the cushion was at 40MB, first run would of passed no problem from the logs in the forums, second run according to his logs here would of triggered the cleanup. (But we know that frees more than 1MB of RAM so he would of flashed successfully). Also the Kernel will always be more responsive than our script. The kernel would near instantly trigger swapping while our script only checks at standard intervals, so I rather give the script as much headroom as possible.

2. Removed the unnecessary Migration Function.
   As we discussed, this next release will drop the migration function, it's been around for months so we should be safe to remove and free some space in the script for logic that will actually be used. I always knew this function I built would be temporary.

3. Added the Toggle option in the Advanced Options for BACKUPMON.
   As we discussed, this next release will give the user control over the BACKUPMON backups. Toggle is placed in the Advanced Options Menu and is hidden if BACKUPMON is not installed. Default Options is still "Enabled". This is incase the user already has daily backups configured and simply wants MerlinAU to handle Auto-Updates.

It's all looking very good. I do like the extra cushion upped to 40MB. As you said, it's safer to give us the extra headroom to prevent any issues when being too close to the kernel watermarks for "low memory" conditions.

@Martinski4GitHub
Copy link
Collaborator

I know I suck, I upped the cushion from 32, to 33 to 35 to 40 lol. Can you tell im indecisive?

No but I'm happy with 40. I thought about it overnight in my dreams. That gives our script double the steps the kernel gives itself. That should be enough headroom for the script to trigger a cleanup if it falls below between the checks.

I don't see it as indecisiveness. You're just trying to come up with a better/safer value based on the limited number of data points we have so far, and given the fact that the built-in function that does the actual F/W Update is essentially a "black box" to us. We have a good idea of what is doing but it's only based on its external behavior; we don't really know its internal mechanism and how it requests RAM pages & manages memory, so we have to be very careful when getting too close to a possible OOM condition.

@ExtremeFiretop
Copy link
Owner Author

ExtremeFiretop commented Mar 21, 2024

I know I suck, I upped the cushion from 32, to 33 to 35 to 40 lol. Can you tell im indecisive?
No but I'm happy with 40. I thought about it overnight in my dreams. That gives our script double the steps the kernel gives itself. That should be enough headroom for the script to trigger a cleanup if it falls below between the checks.

I don't see it as indecisiveness. You're just trying to come up with a better/safer value based on the limited number of data points we have so far, and given the fact that the built-in function that does the actual F/W Update is essentially a "black box" to us. We have a good idea of what is doing but it's only based on its external behavior; we don't really know its internal mechanism and how it requests RAM pages & manages memory, so we have to be very careful when getting too close to a possible OOM condition.

Exactly. The 'black box' to us is the key point.
As you mentioned through external observations we can make some educated guesses on what flow they are using and what the general logic it may be, but asides from that the specifics are unknown so I rather give it as much headroom as possible.

@ExtremeFiretop ExtremeFiretop merged commit f9ec4c7 into dev Mar 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants