Extra packages not restored after image upgrade

[filippocarletti]

Extra package may not be reinstalled after upgrade.
See #1556 for the implementation.

Steps to reproduce

• opkg update && opkg install nmap
• tail -1 /etc/backup/installed_packages.txt
• sysupgrade

Expected behavior

nmap working

Actual behavior

-bash: nmap: command not found

See also

https://docs.nethsecurity.org/en/latest/updates.html#restore-extra-packages-section

Apr 11 10:30:08 firewall.fbfitaly.com ns-restore-extra-packages[8296]: Downloading https://secret/repository/enterprise/nethsecurity/8.7.2/targets/x86/64/packages/Packages.gz
Apr 11 10:30:11 firewall.fbfitaly.com ns-restore-extra-packages[8296]: Updated list of available packages in /var/opkg-lists/nethsecurity_core
Apr 11 10:30:11 firewall.fbfitaly.com ns-restore-extra-packages[8296]: Downloading https://secret/repository/enterprise/nethsecurity/8.7.2/targets/x86/64/packages/Packages.sig
Apr 11 10:30:12 firewall.fbfitaly.com ns-restore-extra-packages[8296]: Signature check passed.
Apr 11 10:30:12 firewall.fbfitaly.com ns-restore-extra-packages[8296]: Downloading https://secret/repository/enterprise/nethsecurity/8.7.2/packages/x86_64/base/Packages.gz
Apr 11 10:30:14 firewall.fbfitaly.com ns-restore-extra-packages[8296]: Updated list of available packages in /var/opkg-lists/nethsecurity_base
Apr 11 10:30:14 firewall.fbfitaly.com ns-restore-extra-packages[8296]: Downloading https://secret/repository/enterprise/nethsecurity/8.7.2/packages/x86_64/base/Packages.sig
Apr 11 10:30:16 firewall.fbfitaly.com ns-restore-extra-packages[8296]: Signature check passed.
Apr 11 10:30:16 firewall.fbfitaly.com ns-restore-extra-packages[8296]: Downloading https://secret/repository/enterprise/nethsecurity/8.7.2/packages/x86_64/luci/Packages.gz
Apr 11 10:30:17 firewall.fbfitaly.com ns-restore-extra-packages[8296]: Updated list of available packages in /var/opkg-lists/nethsecurity_luci
Apr 11 10:30:17 firewall.fbfitaly.com ns-restore-extra-packages[8296]: Downloading https://secret/repository/enterprise/nethsecurity/8.7.2/packages/x86_64/luci/Packages.sig
Apr 11 10:30:19 firewall.fbfitaly.com ns-restore-extra-packages[8296]: Signature check passed.
Apr 11 10:30:19 firewall.fbfitaly.com ns-restore-extra-packages[8296]: Downloading https://secret/repository/enterprise/nethsecurity/8.7.2/packages/x86_64/nethsecurity/Packages.gz
Apr 11 10:30:20 firewall.fbfitaly.com ns-restore-extra-packages[8296]: Updated list of available packages in /var/opkg-lists/nethsecurity_nethsecurity
Apr 11 10:30:20 firewall.fbfitaly.com ns-restore-extra-packages[8296]: Downloading https://secret/repository/enterprise/nethsecurity/8.7.2/packages/x86_64/nethsecurity/Packages.sig
Apr 11 10:30:22 firewall.fbfitaly.com ns-restore-extra-packages[8296]: Signature check passed.
Apr 11 10:30:22 firewall.fbfitaly.com ns-restore-extra-packages[8296]: Downloading https://secret/repository/enterprise/nethsecurity/8.7.2/packages/x86_64/packages/Packages.gz
Apr 11 10:30:23 firewall.fbfitaly.com ns-restore-extra-packages[8296]: Updated list of available packages in /var/opkg-lists/nethsecurity_packages
Apr 11 10:30:23 firewall.fbfitaly.com ns-restore-extra-packages[8296]: Downloading https://secret/repository/enterprise/nethsecurity/8.7.2/packages/x86_64/packages/Packages.sig
Apr 11 10:30:25 firewall.fbfitaly.com ns-restore-extra-packages[8296]: Signature check passed.
Apr 11 10:30:25 firewall.fbfitaly.com ns-restore-extra-packages[8296]: Downloading https://secret/repository/enterprise/nethsecurity/8.7.2/packages/x86_64/routing/Packages.gz
Apr 11 10:30:26 firewall.fbfitaly.com ns-restore-extra-packages[8296]: Updated list of available packages in /var/opkg-lists/nethsecurity_routing
Apr 11 10:30:26 firewall.fbfitaly.com ns-restore-extra-packages[8296]: Downloading https://secret/repository/enterprise/nethsecurity/8.7.2/packages/x86_64/routing/Packages.sig
Apr 11 10:30:28 firewall.fbfitaly.com ns-restore-extra-packages[8296]: Signature check passed.
Apr 11 10:30:33 firewall.fbfitaly.com ns-restore-extra-packages[8296]: Installing nmap (7.95-r1) to root...
Apr 11 10:30:33 firewall.fbfitaly.com ns-restore-extra-packages[8296]: Downloading https://secret/repository/enterprise/nethsecurity/8.7.2/packages/x86_64/packages/nmap_7.95-r1_x86_64.ipk
Apr 11 10:30:39 firewall.fbfitaly.com ns-restore-extra-packages[8296]: Collected errors:
Apr 11 10:30:39 firewall.fbfitaly.com ns-restore-extra-packages[8296]:  * opkg_download: Failed to download https://secret/repository/enterprise/nethsecurity/8.7.2/packages/x86_64/packages/nmap_7.95-r1_x86_64.ipk, wget returned 4.
Apr 11 10:30:39 firewall.fbfitaly.com ns-restore-extra-packages[8296]:  * opkg_download: Check your network settings and connectivity.
Apr 11 10:30:39 firewall.fbfitaly.com ns-restore-extra-packages[8296]: 
Apr 11 10:30:39 firewall.fbfitaly.com ns-restore-extra-packages[8296]:  * opkg_install_pkg: Failed to download nmap. Perhaps you need to run 'opkg update'?
Apr 11 10:30:39 firewall.fbfitaly.com ns-restore-extra-packages[8296]:  * opkg_install_cmd: Cannot install package nmap.
Apr 11 10:30:39 firewall.fbfitaly.com ns-restore-extra-packages[8296]: Restored package: nmap

[gsanchietti]

After further investigation and discussion, we have clarified the root cause of the failure to restore extra packages:

• The initial hypothesis regarding a missing newline in /etc/backup/installed_packages.txt preventing the grep match was debunked. Testing shows that grep -E '\w+\s+overlay$' correctly identifies the package even without a trailing newline.
• The logs show a wget returned 4 (Network failure) error during the execution of ns-restore-extra-packages. This suggests that the script attempted to download the package (nmap) before the WAN connection was fully established or stable after the reboot.
• There is a misleading log entry, which is a cosmetic bug. Even though the download failed, the script logged Restored package: nmap. This is because the script currently doesn't verify the exit code of the installation command effectively.

Proposed solution for next major OpenWrt release (25.x)

• consider making the script more resilient by attempting a reinstall on the next boot if the package is still missing, or implementing a retry loop to account for WAN initialization delays.
• improve the output logic so it only reports "Restored" if the package is actually present, maybe we can relay on apk exit code in next release (opkg exit code is not reliable)

[gsanchietti]

⚠️ The testing is on the dev branch for the upcoming NethSecurity 8.8: apk is required! ⚠️

Testing image: 8.8.0-opkg_dismiss_8.8.20260514141154.3741212f2

Test case

• Disable the internet access
• Verify inside the log that the system logs correctly if a package has failed the installation
• Verify the system tries to reinstall a package if the first installation has failed
• Verify the init.d is disabled when all packages has been restore