Verify system software status before and after the upgrade

[cdot65]

Is your feature request related to a problem? Please describe.
When upgrading PAN-OS on a firewall using the pan-os-upgrade utility, it is important to ensure that all critical system processes are running properly before and after the upgrade. Currently, the utility does not have a built-in mechanism to check the status of system processes, which may lead to upgrades being performed on firewalls with pre-existing issues or upgrades being considered successful even if some processes fail to start after the upgrade.

Describe the solution you'd like
Enhance the pan-os-upgrade utility to include a system software status check before and after the upgrade process. The utility should:

1. Before initiating the upgrade, run the show system software status command on the firewall to retrieve the status of all system processes.
2. Parse the output of the command and check if all critical processes are in the "Running" state.
3. If any critical process is not running before the upgrade, display a warning message to the user and prompt them to either proceed with the upgrade or abort the process.
4. After the upgrade is completed and the firewall is back online, run the show system software status command again.
5. Compare the post-upgrade process status with the pre-upgrade status to ensure that all critical processes are still in the "Running" state.
6. If any critical process is not running after the upgrade, consider the upgrade as failed and display an error message indicating which processes are not running.
7. If all critical processes are running both before and after the upgrade, consider the upgrade as successful.

Describe alternatives you've considered
An alternative approach could be to rely on the existing upgrade status messages provided by the firewall itself. However, these messages may not always cover the status of all critical processes, and relying solely on them might not provide a comprehensive view of the system's health before and after the upgrade.

Additional context
Here are a few additional points to consider:

• The list of critical processes to be checked should be configurable or derived from best practices recommended by Palo Alto Networks.
• The utility should provide clear and informative messages about the status of system processes, both before and after the upgrade.
• In case of any process not running, the utility should log the details and provide troubleshooting information to assist users in resolving the issue.
• Consider adding a flag or configuration option to allow users to skip the system software status check if they prefer to manage it separately.
• Update the project's documentation to include information about this new feature, explaining how it enhances the upgrade process by verifying the system software status.

By implementing this feature, the pan-os-upgrade utility will provide an additional layer of assurance by checking the status of critical system processes before and after the upgrade. This will help identify potential issues early in the upgrade process and ensure that the upgraded firewall is functioning as expected.

[cdot65]

will revisit when time permits, closing for now