Transition systemd service states during upgrade #1060

matejmatuska · 2023-03-15T10:04:01Z

Sometimes after the upgrade some services end up disabled even if they have been enabled on the source system.

There are already two separate actors that fix this for device_cio_free.service and rsyncd.service.

A new actor transition-systemd-services-states handles this generally for all services. A "desired" state is determined depending on state and vendor preset of both source and target system and a SystemdServicesTasks message is produced with each service that isn't already in the "desired" state.

This obsoletes #1043 and TODO TEST #977 and closes #826

NOTE: In the future, if there are SystemdServicesTasks produced to with different state than the one produced by the transition-systemd-services-states actor, the setsystemdservicesstates actor will raise an inhibitor. The solution of this is out of the scope of this task, but some of the possible solutions are:

Introduce a priority for each task
Change the produced model of transition-systemd-services-states to 'Automatic/DefaultSystemdServicesTasks' and treat it with lower priority in setsystemdservicesstates

Jira ref.: OAMG-1745

github-actions · 2023-03-15T10:04:14Z

Thank you for contributing to the Leapp project!

Please note that every PR needs to comply with the Leapp Guidelines and must pass all tests in order to be mergeable.
If you want to request a review or rebuild a package in copr, you can use following commands as a comment:

review please to notify leapp developers of review request
/packit copr-build to submit a public copr build using packit

To launch regression testing public members of oamg organization can leave the following comment:

/rerun to schedule basic regression tests using this pr build and leapp*master* as artifacts
/rerun 42 to schedule basic regression tests using this pr build and leapp*PR42* as artifacts
/rerun-sst to schedule sst tests using this pr build and leapp*master* as artifacts
/rerun-sst 42 to schedule sst tests using this pr build and leapp*PR42* as artifacts

Please open ticket in case you experience technical problem with the CI. (RH internal only)

Note: In case there are problems with tests not being triggered automatically on new PR/commit or pending for a long time, please consider rerunning the CI by commenting leapp-ci build (might require several comments). If the problem persists, contact leapp-infra.

matejmatuska · 2023-03-15T10:21:09Z

Tested with on 7->8 with:

rsyncd.service enabled, with actor from Keep rsyncd.service enabled after IPU #1043 removed
dhcpd.service enabled (see dhcp server unit file is changed to disabled after upgrade #826)

After the reboot:

Produced message:

root@leapp-20230314094848 ~]# leapp-inspector messages --type SystemdServicesTasks
######################################################################
                          PRODUCED MESSAGES
######################################################################
Stamp: 2023-03-14T21:42:24.318449Z
Actor: transition_systemd_services_states
Phase: Applications
Type: SystemdServicesTasks
Message_data:
{
    "to_disable": [],
    "to_enable": [
        "abrt-ccpp.service",
        "dhcpd.service",
        "rsyncd.service"
    ]
}
######################################################################

Services are correctly kept enabled

[root@leapp-20230314094848 ~]# systemctl is-enabled rsyncd
enabled
[root@leapp-20230314094848 ~]# systemctl is-enabled dhcpd
enabled

matejmatuska · 2023-03-15T15:21:09Z

Ran the same tests after the refactor and addition of reports and the results are the same.

pirat89 · 2023-04-18T11:41:01Z

@matejmatuska finally on my todolist for tomorrow!! :)

pirat89 · 2023-04-25T10:09:27Z

@matejmatuska finally on my todolist for tomorrow!! :)

bad @pirat89 !! 🧛‍♂️ jumping on the review now

pirat89 · 2023-04-25T10:52:52Z

/packit build

pirat89 · 2023-08-25T09:08:51Z

@matejmatuska rebase please

.../actors/systemd/transitionsystemdservicesstates/libraries/transitionsystemdservicesstates.py

pirat89 · 2023-10-06T12:55:44Z

/packit build

pirat89 · 2023-10-06T14:36:32Z

Seems to me working as expected, tested 7 -> 8 and 8 -> 9, tested:

preserve enabled services (rsyncd)
preserved explicitely disabled services (preset: enabled, manually disabled) (kdump)

@matejmatuska squash please fix commits, then we can merge it.

matejmatuska · 2023-10-06T14:44:51Z

Seems to me working as expected, tested 7 -> 8 and 8 -> 9, tested:

preserve enabled services (rsyncd)

preserved explicitely disabled services (preset: enabled, manually disabled) (kdump)

@matejmatuska squash please fix commits, then we can merge it.

Would you prefer to keep the "remove obsoleted rsyncd.." separate from the main commit?

pirat89 · 2023-10-09T07:48:23Z

@matejmatuska you can keep it separated if you want

matejmatuska · 2023-10-09T16:23:37Z

@matejmatuska you can keep it separated if you want

@pirat89 Done, sorry for the delay.

matejmatuska · 2023-10-09T16:39:57Z

I am not sure what's wrong with the codespell and linter, the warnings seem to be unrelated to this PR.

pirat89 · 2023-10-10T12:52:06Z

@matejmatuska it's caused by newly released versions. opening PR with the fix (002308d)

Sometimes after the upgrade some services end up disabled even if they have been enabled on the source system. There are already two separate actors that fix this for `device_cio_free.service` and `rsyncd.service`. A new actor `transition-systemd-services-states` handles this generally for all services. A "desired" state is determined depending on state and vendor preset of both source and target system and a SystemdServicesTasks message is produced with each service that isn't already in the "desired" state. Jira ref.: OAMG-1745

The `transitionsystemdservicesstates` actor now handles all such services generically, which makes this actor obsolete.

pirat89

Finally it's here \o/ merging. Thanks!!

Note: tests passed. these forever queued tests are optional and not expected now to be executed. the fix is in progress.

## Packaging - Requires xfsprogs and e2fsprogs (oamg#1154) - Bump leapp-repository-dependencies to 10 (oamg#1154) ## Upgrade handling ### Fixes - Detect changes in openssl default configuration file and restore it to the default to the target default during the upgrade to reduce risk of potential issues (oamg#1131) - Do not try to download data files anymore when missing as the service is obsoleted since the data is part of installed packages (oamg#1120) - Drop the invalid `tuv` target channel (oamg#1130) - Fix handling of symlinks under /etc/pki when managing certificates (oamg#1135, oamg#1160, oamg#1166) - Fix semanage import issue (oamg#1164) - Fix the issue of going out of bounds in the isccfg parser (oamg#1124) - Fix traceback when saving the rhsm facts results and the /etc/rhsm/facts directory doesn’t exist yet (oamg#1132) - Handle the upgrade better when a proxy is configured in YUM/DNF configutations (oamg#1143) - Load all rpm repository substitutions that dnf knows about, not just releasever since repofiles may use the other substitutions too (oamg#1134) - Minor updates of generated reports (oamg#1151) - Print nice error msg when device and driver deprecation data is malformed (oamg#1168) - Report information about required manual steps after the upgrade when openssl-ibmca is installed (oamg#1131) - Update error messages and reports when installed upgrade data files are malformed or missing (oamg#1120) - [IPU 7 -> 8] Fix the upgrade of the RH Satellite server when tomcat is installed (oamg#1150) - [IPU 8 -> 9] Fix the upgrade from RHEL 8.9+ when the release is locked by subscription-manager (oamg#1136, oamg#1138) ### Enhancements - Update upgrade paths: (oamg#1146, oamg#1147, oamg#1175) - RHEL 7.9 -> 8.10, 8.8 (default: 8.10) - RHEL with SAPAHA 7.9 -> 8.10, 8.8 (default: 8.8) - RHEL w/o SAP HANA 8.8 -> 9.2 - RHEL w/o SAP HANA 8.10 -> 9.4 - Added possibility to define DNF configuration for the target system (oamg#1143) - Code cleaning: drop redundant and invalid NFS checks (oamg#1127) - Default to NO_RHSM mode when subscription-manager is not found (oamg#1133) - Detect customized configuration of dynamic linker (oamg#1118) - Detect possible unexpected RPM GPG keys has been installed during RPM transaction (oamg#1101) - Drop obsoleted upgrade paths that relates to releases: 8.6, 8.9, 9.0, 9.3 (oamg#1175) - Ignore Leapp related PES events (oamg#1153) - Introduce generic transition of systemd services states during the IPU (oamg#1060, oamg#1174) - Introduce possibility to upgrade with local repositories (oamg#1099) - Introduced some changes getting us closer to possibility of IPU for Centos (Stream) systems (oamg#1140) - Report the upgrade customisations and modifications of the upgrade tooling (oamg#1148) - Simplify handling of upgrades on systems using RHUI, reducing the maintenance burden for cloud providers (oamg#1057) - Update the leapp upgrade data files - bump data stream to "3.0" (oamg#1163, oamg#1165, oamg#1170) - [IPU 8 -> 9] Enable upgrades RHEL 8 -> 9 using RHUI on Alibaba cloud (oamg#1137, oamg#1165, oamg#1172) ## Additional changes interesting for devels - Introduced new functions returning a list of packages related to upgrade - see the rpms library (oamg#1156) - Make detection of installed signed packages distribution agnostic - covers RHEL & CentOS (oamg#876) - Model InstalledRedHatSignedRPM is deprecated, replaced by DistributionSignedRPM (oamg#876)

## Packaging - Requires xfsprogs and e2fsprogs (oamg#1154) - Bump leapp-repository-dependencies to 10 (oamg#1154) ## Upgrade handling ### Fixes - Detect changes in openssl default configuration file and restore it to the default to the target default during the upgrade to reduce risk of potential issues (oamg#1131) - Do not try to download data files anymore when missing as the service is obsoleted since the data is part of installed packages (oamg#1120) - Drop the invalid `tuv` target channel (oamg#1130) - Fix handling of symlinks under /etc/pki when managing certificates (oamg#1135, oamg#1160, oamg#1166) - Fix semanage import issue (oamg#1164) - Fix the issue of going out of bounds in the isccfg parser (oamg#1124) - Fix traceback when saving the rhsm facts results and the /etc/rhsm/facts directory doesn’t exist yet (oamg#1132) - Handle the upgrade better when a proxy is configured in YUM/DNF configutations (oamg#1143) - Load all rpm repository substitutions that dnf knows about, not just releasever since repofiles may use the other substitutions too (oamg#1134) - Minor updates of generated reports (oamg#1151) - Print nice error msg when device and driver deprecation data is malformed (oamg#1168) - Report information about required manual steps after the upgrade when openssl-ibmca is installed (oamg#1131) - Update error messages and reports when installed upgrade data files are malformed or missing (oamg#1120) - [IPU 7 -> 8] Fix the upgrade of the RH Satellite server when tomcat is installed (oamg#1150) - [IPU 8 -> 9] Fix the upgrade from RHEL 8.9+ when the release is locked by subscription-manager (oamg#1136, oamg#1138) ### Enhancements - Update upgrade paths: (oamg#1146, oamg#1147, oamg#1175) - RHEL 7.9 -> 8.10, 8.8 (default: 8.10) - RHEL with SAPAHA 7.9 -> 8.10, 8.8 (default: 8.8) - RHEL w/o SAP HANA 8.8 -> 9.2 - RHEL w/o SAP HANA 8.10 -> 9.4 - Added possibility to define DNF configuration for the target system (oamg#1143) - Code cleaning: drop redundant and invalid NFS checks (oamg#1127) - Default to NO_RHSM mode when subscription-manager is not found (oamg#1133) - Detect customized configuration of dynamic linker (oamg#1118) - Detect possible unexpected RPM GPG keys has been installed during RPM transaction (oamg#1101) - Drop obsoleted upgrade paths that relates to releases: 8.6, 8.9, 9.0, 9.3 (oamg#1175) - Ignore Leapp related PES events (oamg#1153) - Introduce generic transition of systemd services states during the IPU (oamg#1060, oamg#1174) - Introduce possibility to upgrade with local repositories (oamg#1099) - Introduced some changes getting us closer to possibility of IPU for Centos (Stream) systems (oamg#1140) - Report the upgrade customisations and modifications of the upgrade tooling (oamg#1148) - Simplify handling of upgrades on systems using RHUI, reducing the maintenance burden for cloud providers (oamg#1057) - Update the leapp upgrade data files - bump data stream to "3.0" (oamg#1163, oamg#1165, oamg#1170) - [IPU 8 -> 9] Enable upgrades RHEL 8 -> 9 using RHUI on Alibaba cloud (oamg#1137, oamg#1165, oamg#1172) - Unify breakpoints inside the upgrade initramfs for the easier troubleshooting (oamg#1157) ## Additional changes interesting for devels - Introduced new functions returning a list of packages related to upgrade - see the rpms library (oamg#1156) - Make detection of installed signed packages distribution agnostic - covers RHEL & CentOS (oamg#876) - Model InstalledRedHatSignedRPM is deprecated, replaced by DistributionSignedRPM (oamg#876)

## Packaging - Requires xfsprogs and e2fsprogs (#1154) - Bump leapp-repository-dependencies to 10 (#1154) ## Upgrade handling ### Fixes - Detect changes in openssl default configuration file and restore it to the default to the target default during the upgrade to reduce risk of potential issues (#1131) - Do not try to download data files anymore when missing as the service is obsoleted since the data is part of installed packages (#1120) - Drop the invalid `tuv` target channel (#1130) - Fix handling of symlinks under /etc/pki when managing certificates (#1135, #1160, #1166) - Fix semanage import issue (#1164) - Fix the issue of going out of bounds in the isccfg parser (#1124) - Fix traceback when saving the rhsm facts results and the /etc/rhsm/facts directory doesn’t exist yet (#1132) - Handle the upgrade better when a proxy is configured in YUM/DNF configutations (#1143) - Load all rpm repository substitutions that dnf knows about, not just releasever since repofiles may use the other substitutions too (#1134) - Minor updates of generated reports (#1151) - Print nice error msg when device and driver deprecation data is malformed (#1168) - Report information about required manual steps after the upgrade when openssl-ibmca is installed (#1131) - Update error messages and reports when installed upgrade data files are malformed or missing (#1120) - [IPU 7 -> 8] Fix the upgrade of the RH Satellite server when tomcat is installed (#1150) - [IPU 8 -> 9] Fix the upgrade from RHEL 8.9+ when the release is locked by subscription-manager (#1136, #1138) ### Enhancements - Update upgrade paths: (#1146, #1147, #1175) - RHEL 7.9 -> 8.10, 8.8 (default: 8.10) - RHEL with SAPAHA 7.9 -> 8.10, 8.8 (default: 8.8) - RHEL w/o SAP HANA 8.8 -> 9.2 - RHEL w/o SAP HANA 8.10 -> 9.4 - Added possibility to define DNF configuration for the target system (#1143) - Code cleaning: drop redundant and invalid NFS checks (#1127) - Default to NO_RHSM mode when subscription-manager is not found (#1133) - Detect customized configuration of dynamic linker (#1118) - Detect possible unexpected RPM GPG keys has been installed during RPM transaction (#1101) - Drop obsoleted upgrade paths that relates to releases: 8.6, 8.9, 9.0, 9.3 (#1175) - Ignore Leapp related PES events (#1153) - Introduce generic transition of systemd services states during the IPU (#1060, #1174) - Introduce possibility to upgrade with local repositories (#1099) - Introduced some changes getting us closer to possibility of IPU for Centos (Stream) systems (#1140) - Report the upgrade customisations and modifications of the upgrade tooling (#1148) - Simplify handling of upgrades on systems using RHUI, reducing the maintenance burden for cloud providers (#1057) - Update the leapp upgrade data files - bump data stream to "3.0" (#1163, #1165, #1170) - [IPU 8 -> 9] Enable upgrades RHEL 8 -> 9 using RHUI on Alibaba cloud (#1137, #1165, #1172) - Unify breakpoints inside the upgrade initramfs for the easier troubleshooting (#1157) ## Additional changes interesting for devels - Introduced new functions returning a list of packages related to upgrade - see the rpms library (#1156) - Make detection of installed signed packages distribution agnostic - covers RHEL & CentOS (#876) - Model InstalledRedHatSignedRPM is deprecated, replaced by DistributionSignedRPM (#876)

matejmatuska force-pushed the transition-systemd-services-states branch 9 times, most recently from e767206 to 5c55f35 Compare March 15, 2023 15:11

matejmatuska marked this pull request as ready for review March 15, 2023 15:21

matejmatuska force-pushed the transition-systemd-services-states branch from 20143ff to cab4818 Compare March 15, 2023 15:41

pirat89 added this to the 8.9/9.3 milestone Apr 25, 2023

pirat89 added the enhancement New feature or request label Apr 25, 2023

matejmatuska force-pushed the transition-systemd-services-states branch from cab4818 to d22f0bd Compare July 28, 2023 12:09

pirat89 self-assigned this Jul 28, 2023

pirat89 modified the milestones: 8.9/9.3, 7.9/8.10 Jul 31, 2023

matejmatuska force-pushed the transition-systemd-services-states branch from d22f0bd to 8183ab6 Compare August 28, 2023 13:46

matejmatuska force-pushed the transition-systemd-services-states branch from 8183ab6 to 6ddf86e Compare September 7, 2023 14:10

matejmatuska assigned matejmatuska and pirat89 and unassigned pirat89 and matejmatuska Sep 7, 2023

pirat89 reviewed Sep 8, 2023

View reviewed changes

.../actors/systemd/transitionsystemdservicesstates/libraries/transitionsystemdservicesstates.py Outdated Show resolved Hide resolved

fernflower reviewed Sep 11, 2023

View reviewed changes

.../actors/systemd/transitionsystemdservicesstates/libraries/transitionsystemdservicesstates.py Outdated Show resolved Hide resolved

matejmatuska force-pushed the transition-systemd-services-states branch from 397a6f9 to 99ea54a Compare October 9, 2023 16:22

matejmatuska force-pushed the transition-systemd-services-states branch from 99ea54a to fae327d Compare October 9, 2023 16:31

matejmatuska force-pushed the transition-systemd-services-states branch from fae327d to 51307b6 Compare October 10, 2023 13:59

matejmatuska added 2 commits October 18, 2023 14:15

Remove obsoleted enablersyncdservice actor

13f329c

The `transitionsystemdservicesstates` actor now handles all such services generically, which makes this actor obsolete.

matejmatuska force-pushed the transition-systemd-services-states branch from 51307b6 to 13f329c Compare October 18, 2023 12:16

pirat89 added the changelog-checked The merger/reviewer checked the changelog draft document and updated it when relevant label Oct 19, 2023

pirat89 approved these changes Oct 19, 2023

View reviewed changes

pirat89 merged commit bea0f89 into oamg:master Oct 19, 2023
17 of 29 checks passed

pirat89 mentioned this pull request Feb 13, 2024

Release 0.20.0 #1177

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Transition systemd service states during upgrade #1060

Transition systemd service states during upgrade #1060

matejmatuska commented Mar 15, 2023 •

edited

github-actions bot commented Mar 15, 2023

matejmatuska commented Mar 15, 2023 •

edited

matejmatuska commented Mar 15, 2023

pirat89 commented Apr 18, 2023

pirat89 commented Apr 25, 2023

pirat89 commented Apr 25, 2023

pirat89 commented Aug 25, 2023

pirat89 commented Oct 6, 2023

pirat89 commented Oct 6, 2023 •

edited

matejmatuska commented Oct 6, 2023

pirat89 commented Oct 9, 2023

matejmatuska commented Oct 9, 2023

matejmatuska commented Oct 9, 2023

pirat89 commented Oct 10, 2023 •

edited

pirat89 left a comment •

edited

Transition systemd service states during upgrade #1060

Transition systemd service states during upgrade #1060

Conversation

matejmatuska commented Mar 15, 2023 • edited

github-actions bot commented Mar 15, 2023

Thank you for contributing to the Leapp project!

matejmatuska commented Mar 15, 2023 • edited

After the reboot:

matejmatuska commented Mar 15, 2023

pirat89 commented Apr 18, 2023

pirat89 commented Apr 25, 2023

pirat89 commented Apr 25, 2023

pirat89 commented Aug 25, 2023

pirat89 commented Oct 6, 2023

pirat89 commented Oct 6, 2023 • edited

matejmatuska commented Oct 6, 2023

pirat89 commented Oct 9, 2023

matejmatuska commented Oct 9, 2023

matejmatuska commented Oct 9, 2023

pirat89 commented Oct 10, 2023 • edited

pirat89 left a comment • edited

Choose a reason for hiding this comment

matejmatuska commented Mar 15, 2023 •

edited

matejmatuska commented Mar 15, 2023 •

edited

pirat89 commented Oct 6, 2023 •

edited

pirat89 commented Oct 10, 2023 •

edited

pirat89 left a comment •

edited