BZ#2142270 - reindex the database before the installer runs #1018

evgeni · 2023-01-27T14:00:03Z

No description provided.

github-actions · 2023-01-27T14:00:19Z

Thank you for contributing to the Leapp project!

Please note that every PR needs to comply with the Leapp Guidelines and must pass all tests in order to be mergeable.
If you want to request a review or rebuild a package in copr, you can use following commands as a comment:

review please to notify leapp developers of review request
/packit copr-build to submit a public copr build using packit

To launch regression testing public members of oamg organization can leave the following comment:

/rerun to schedule basic regression tests using this pr build and leapp*master* as artifacts
/rerun 42 to schedule basic regression tests using this pr build and leapp*PR42* as artifacts
/rerun-sst to schedule sst tests using this pr build and leapp*master* as artifacts
/rerun-sst 42 to schedule sst tests using this pr build and leapp*PR42* as artifacts

Please open ticket in case you experience technical problem with the CI. (RH internal only)

Note: In case there are problems with tests not being triggered automatically on new PR/commit or pending for a long time, please consider rerunning the CI by commenting leapp-ci build (might require several comments). If the problem persists, contact leapp-infra.

fernflower · 2023-01-27T14:30:32Z

/rerun

fernflower

One minor comment, otherwise LGTM

fernflower · 2023-01-27T14:57:25Z

repos/system_upgrade/el7toel8/actors/satellite_upgrader/actor.py

+                run(['sed', '-i', '/data_directory/d', '/var/lib/pgsql/data/postgresql.conf'])
+                run(['systemctl', 'start', 'postgresql'])
+                run(['runuser', '-u', 'postgres', '--', 'reindexdb', '-a'])
+            except (OSError, CalledProcessError) as e:


From my understanding you might get CalledProcessError during run execution, not OSError

Yeah, I think I had it for the installer run because in some intermediate version of the first actor I managed to end up in a situation where the installer binary would not be there and that yieleded a OSError, can drop because that should really not happen here

@fernflower yes, the run function can returns both, OSError and CalledProcessError. The second one happens e.g. when the process exits with non-zero exit code. The first one in case the command cannot be called (e.g. due to missing executable file, but also permissions could result in this issue if I remember well - not sure about that one). So it's usually better to catch both, even when for the core utilities it's expected they are always present. In this case I consider it ok to catch just CalledProcessError, however, having them both is additional seatbelt.

fernflower · 2023-01-27T14:57:53Z

/rerun

github-actions · 2023-01-27T15:02:35Z

Copr build succeeded: https://copr.fedorainfracloud.org/coprs/build/5328828

github-actions · 2023-01-27T15:23:36Z

Testing Farm request for RHEL-8.6-rhui/5328828 regression testing has been created.
Once finished, results should be available here.
Full pipeline log.

github-actions · 2023-01-27T15:30:17Z

Testing Farm request for RHEL-7.9-rhui/5328828 regression testing has been created.
Once finished, results should be available here.
Full pipeline log.

github-actions · 2023-01-27T17:08:13Z

Testing Farm request for RHEL-8.6.0-Nightly/5328828 regression testing has been created.
Once finished, results should be available here.
Full pipeline log.

github-actions · 2023-01-27T17:22:36Z

Testing Farm request for RHEL-8.7.0-Nightly/5328828 regression testing has been created.
Once finished, results should be available here.
Full pipeline log.

github-actions · 2023-01-27T17:28:26Z

Testing Farm request for RHEL-7.9-ZStream/5328828 regression testing has been created.
Once finished, results should be available here.
Full pipeline log.

github-actions · 2023-01-27T17:29:51Z

Testing Farm request for RHEL-7.9-ZStream/5328828 regression testing has been created.
Once finished, results should be available here.
Full pipeline log.

pirat89

seems good, but I want to see the confirmation from QE the fix works as expected before the merge.

lpramuk · 2023-01-30T09:33:12Z

Successfully tested using these COPR builds:

leapp-0.15.0-100.20230127140319435999.master.19.g7c551bc.el7.noarch
leapp-upgrade-el7toel8-0.17.0-0.202301271458Z.8c014f1.HEAD.PR1018.el7.noarch

Command:
# leapp upgrade --target 8.6

Logs:


2023-01-27 16:41:11.637 INFO     PID: 6159 leapp.workflow.FirstBoot: Executing actor satellite_upgrader
2023-01-27 16:41:11.732 DEBUG    PID: 7063 leapp.workflow.FirstBoot.satellite_upgrader: External command has started: ['sed', '-i', '/data_directory/d', '/var/lib/pgsql/data/postgresql.conf']
2023-01-27 16:41:11.768 DEBUG    PID: 7063 leapp.workflow.FirstBoot.satellite_upgrader: External command has finished: ['sed', '-i', '/data_directory/d', '/var/lib/pgsql/data/postgresql.conf']
2023-01-27 16:41:11.775 DEBUG    PID: 7063 leapp.workflow.FirstBoot.satellite_upgrader: External command has started: ['systemctl', 'start', 'postgresql']
2023-01-27 16:41:11.967 DEBUG    PID: 7063 leapp.workflow.FirstBoot.satellite_upgrader: External command has finished: ['systemctl', 'start', 'postgresql']
2023-01-27 16:41:11.973 DEBUG    PID: 7063 leapp.workflow.FirstBoot.satellite_upgrader: External command has started: ['runuser', '-u', 'postgres', '--', 'reindexdb', '-a']
2023-01-27 16:41:12.56  DEBUG    PID: 7063 leapp.workflow.FirstBoot.satellite_upgrader: reindexdb: reindexing database "candlepin"
2023-01-27 16:41:13.965 DEBUG    PID: 7063 leapp.workflow.FirstBoot.satellite_upgrader: reindexdb: reindexing database "foreman"
2023-01-27 16:41:26.211 DEBUG    PID: 7063 leapp.workflow.FirstBoot.satellite_upgrader: reindexdb: reindexing database "postgres"
2023-01-27 16:41:26.771 DEBUG    PID: 7063 leapp.workflow.FirstBoot.satellite_upgrader: reindexdb: reindexing database "pulpcore"
2023-01-27 16:41:47.129 DEBUG    PID: 7063 leapp.workflow.FirstBoot.satellite_upgrader: reindexdb: reindexing database "template1"
2023-01-27 16:41:47.793 DEBUG    PID: 7063 leapp.workflow.FirstBoot.satellite_upgrader: External command has finished: ['runuser', '-u', 'postgres', '--', 'reindexdb', '-a']
2023-01-27 16:41:47.800 DEBUG    PID: 7063 leapp.workflow.FirstBoot.satellite_upgrader: External command has started: ['foreman-installer', '--disable-system-checks']
...
2023-01-27 16:48:57.286 DEBUG    PID: 7063 leapp.workflow.FirstBoot.satellite_upgrader: External command has finished: ['foreman-installer', '--disable-system-checks']

pirat89

merging

## Packaging - Requires cpio (oamg#979) - Requires python3-gobject-base, NetworkManager-libnm (oamg#969) - Bump leapp-repository-dependencies to 9 (oamg#969, oamg#979) ## Upgrade handling ### Fixes - Add leapp RHUI packages to an allowlist to drop confusing reports (oamg#995) - Check only mounted XFS partitions (oamg#1027) - Detect the kernel-core RPM instead of kernel to prevent an error during post-upgrade phases (oamg#1024) - Disable the amazon-id DNF plugin on AWS during the upgrade stage to omit error messages during the upgrade process caused by missing network connection (oamg#990) - Do not create new *pyc files when running leapp after the DNF upgrade transaction (oamg#1017) - Enable upgrades on s390x when /boot is part of rootfs (oamg#991) - Extend the allow list of RHUI clients by azure-sap-apps to omit confusing report (oamg#974) - Filter out PES events unrelated for the used upgrade path and handle overlapping events (oamg#1008) - Fix scan of ceph volumes on systems without ceph-osd (oamg#1011) - Fix scan of ceph volumes when ceph-osd container is not found (oamg#986) - Fix systemd symlinks that become incorrect during the IPU (oamg#972) - Fix the check of memory (RAM) limits (oamg#984) - Fix the upgrade of IBM Z machines configured with ZFCP (oamg#983) - Ignore external accounts in /etc/passwd (oamg#958) - Inhibit the upgrade when entries in /etc/fstab cause overshadowing during the upgrade (oamg#1009) - Prevent leapp failures caused by re-run of leapp in the upgrade initramfs after previous failure, which causes additional confusing error message hiding original bugs (oamg#996) - Prevent the upgrade with RHSM when a baseos and an appstream target repositories are not discovered (oamg#1001) - RHUI(Azure) handle correctly various SAP images (oamg#1037) - Rework the network configuration handling and parse the configuration data properly (oamg#969) - Set RHSM release for non-ga and non-beta channels (oamg#1033) - Use the "grub" library to find the GRUB device (oamg#989) - [IPU 7 -> 8] Detect corrupted grubenv file (oamg#1012) - [IPU 7 -> 8] Ensure that rsyncd stays enabled if it is enabled prior the upgrade(oamg#1043) - [IPU 7 -> 8] Ensure that satellite metapackages are installed after the upgrade (oamg#994) - [IPU 7 -> 8] Ensure the device_cio_free service stays enabled on s390x after the upgrade (oamg#977) - [IPU 7 -> 8] Fixed checks for RHEL SAP IPU 7.9 -> 8.6 (oamg#978) - [IPU 7 -> 8] Fixed migration of ntp to chrony when the ntp service is masked (oamg#966) - [IPU 7 -> 8] Prevent the traceback during migration of sendmail configuration files when the package is not installed (oamg#1041) - [IPU 7 -> 8] Satellite: reindex all related databases to fix issues due to new locales in RHEL 8 (oamg#1007, oamg#1018) - [IPU 7 -> 8] Use the correct domain name in SSSD reports (oamg#1040) - [IPU 8 -> 9] Added checks for RHEL SAP IPU 8.6 -> 9.0 (oamg#978) - [IPU 8 -> 9] CheckVDO: Ask user for the confirmation only on failures and undetermined devices (oamg#961) - [IPU 8 -> 9] Fix the kernel detection during initramfs creation for new kernel on RHEL 9.2+ (oamg#1048) - [IPU 8 -> 9] Fix the upgrade on Azure using RHUI for SAP Apps images (oamg#975) - [IPU 8 -> 9] Handle correctly firewalld version 0.8 (oamg#963) ### Enhancements - Set new upgrade paths (oamg#988): -- RHEL 7.9 -> 8.8, 8.6 (default: 8.8) -- RHEL 8.6 -> 9.0 -- RHEL 8.8 -> 9.2 - Check that used leapp data are valid and compatible with the installed leapp-repository (oamg#1003) - Detect a proxy configuration in YUM/DNF and adjust an error msg on issues caused by the configuration (oamg#914) - Detect and report systemd symlinks that are broken before the upgrade (oamg#972) - Drop obsoleted upgrade paths (oamg#1047) - Improve remediation instructions for packages in unknown repositories (oamg#1010) - Improve the error message to guide users when discovered more space is needed (oamg#956) - Introduce --nogpgcheck option to skip checking of RPM signatures (oamg#910) - Introduced an option to use an ISO file as a target RHEL version content source (oamg#979) - Introduced possibility to specify what systemd services should be enabled/disabled on the upgraded system (oamg#964) - Map the target repositories also based on the installed content (oamg#967) - Provide common information about systemd services (oamg#959) - Register subscribed systems automatically to Red Hat Insights unless --no-insights-register is used (oamg#1000) - Remove obsoleted GPG keys provided by RH after the upgrade to prevent errors (oamg#1022) - Run upgrade process with checking RPM signatures by default (oamg#910, oamg#993, oamg#1025) - Save breadcrumbs results as RHSM facts (oamg#1002) - Small improvements in various reports (oamg#1038, oamg#1039, oamg#1032) - [IPU 8 -> 9] Detect CIFS also when upgrading from RHEL8 to RHEL9 (PR1035) - [IPU 8 -> 9] Detect RoCE on IBM Z machines and check the configuration is safe for the upgrade (oamg#1030) - [IPU 8 -> 9] Enable upgrades of RHEL 8 for SAP HANA to RHEL 9 on ppc64le (oamg#1042) - [IPU 8 -> 9] Improve the handling of blocklisted certificates (oamg#992) ## Additional changes interesting for devels - Started work on bringing up networking inside the upgrade initramfs - currently available for testing and development purposes when LEAPP_DEVEL_INITRAM_NETWORK is set (oamg#960) - Add leapp debug tools to the upgrade initramfs - dracut upgrade module (oamg#997) - Enable disabling of DNF plugins in the dnfplugin library (oamg#990)

## Packaging - Requires cpio (#979) - Requires python3-gobject-base, NetworkManager-libnm (#969) - Bump leapp-repository-dependencies to 9 (#969, #979) ## Upgrade handling ### Fixes - Add leapp RHUI packages to an allowlist to drop confusing reports (#995) - Check only mounted XFS partitions (#1027) - Detect the kernel-core RPM instead of kernel to prevent an error during post-upgrade phases (#1024) - Disable the amazon-id DNF plugin on AWS during the upgrade stage to omit error messages during the upgrade process caused by missing network connection (#990) - Do not create new *pyc files when running leapp after the DNF upgrade transaction (#1017) - Enable upgrades on s390x when /boot is part of rootfs (#991) - Extend the allow list of RHUI clients by azure-sap-apps to omit confusing report (#974) - Filter out PES events unrelated for the used upgrade path and handle overlapping events (#1008) - Fix scan of ceph volumes on systems without ceph-osd (#1011) - Fix scan of ceph volumes when ceph-osd container is not found (#986) - Fix systemd symlinks that become incorrect during the IPU (#972) - Fix the check of memory (RAM) limits (#984) - Fix the upgrade of IBM Z machines configured with ZFCP (#983) - Ignore external accounts in /etc/passwd (#958) - Inhibit the upgrade when entries in /etc/fstab cause overshadowing during the upgrade (#1009) - Prevent leapp failures caused by re-run of leapp in the upgrade initramfs after previous failure, which causes additional confusing error message hiding original bugs (#996) - Prevent the upgrade with RHSM when a baseos and an appstream target repositories are not discovered (#1001) - RHUI(Azure) handle correctly various SAP images (#1037) - Rework the network configuration handling and parse the configuration data properly (#969) - Set RHSM release for non-ga and non-beta channels (#1033) - Use the "grub" library to find the GRUB device (#989) - [IPU 7 -> 8] Detect corrupted grubenv file (#1012) - [IPU 7 -> 8] Ensure that rsyncd stays enabled if it is enabled prior the upgrade(#1043) - [IPU 7 -> 8] Ensure that satellite metapackages are installed after the upgrade (#994) - [IPU 7 -> 8] Ensure the device_cio_free service stays enabled on s390x after the upgrade (#977) - [IPU 7 -> 8] Fixed checks for RHEL SAP IPU 7.9 -> 8.6 (#978) - [IPU 7 -> 8] Fixed migration of ntp to chrony when the ntp service is masked (#966) - [IPU 7 -> 8] Prevent the traceback during migration of sendmail configuration files when the package is not installed (#1041) - [IPU 7 -> 8] Satellite: reindex all related databases to fix issues due to new locales in RHEL 8 (#1007, #1018) - [IPU 7 -> 8] Use the correct domain name in SSSD reports (#1040) - [IPU 8 -> 9] Added checks for RHEL SAP IPU 8.6 -> 9.0 (#978) - [IPU 8 -> 9] CheckVDO: Ask user for the confirmation only on failures and undetermined devices (#961) - [IPU 8 -> 9] Fix the kernel detection during initramfs creation for new kernel on RHEL 9.2+ (#1048) - [IPU 8 -> 9] Fix the upgrade on Azure using RHUI for SAP Apps images (#975) - [IPU 8 -> 9] Handle correctly firewalld version 0.8 (#963) ### Enhancements - Set new upgrade paths (#988): -- RHEL 7.9 -> 8.8, 8.6 (default: 8.8) -- RHEL 8.6 -> 9.0 -- RHEL 8.8 -> 9.2 - Check that used leapp data are valid and compatible with the installed leapp-repository (#1003) - Detect a proxy configuration in YUM/DNF and adjust an error msg on issues caused by the configuration (#914) - Detect and report systemd symlinks that are broken before the upgrade (#972) - Drop obsoleted upgrade paths (#1047) - Improve remediation instructions for packages in unknown repositories (#1010) - Improve the error message to guide users when discovered more space is needed (#956) - Introduce --nogpgcheck option to skip checking of RPM signatures (#910) - Introduced an option to use an ISO file as a target RHEL version content source (#979) - Introduced possibility to specify what systemd services should be enabled/disabled on the upgraded system (#964) - Map the target repositories also based on the installed content (#967) - Provide common information about systemd services (#959) - Register subscribed systems automatically to Red Hat Insights unless --no-insights-register is used (#1000) - Remove obsoleted GPG keys provided by RH after the upgrade to prevent errors (#1022) - Run upgrade process with checking RPM signatures by default (#910, #993, #1025) - Save breadcrumbs results as RHSM facts (#1002) - Small improvements in various reports (#1038, #1039, #1032) - [IPU 8 -> 9] Detect CIFS also when upgrading from RHEL8 to RHEL9 (PR1035) - [IPU 8 -> 9] Detect RoCE on IBM Z machines and check the configuration is safe for the upgrade (#1030) - [IPU 8 -> 9] Enable upgrades of RHEL 8 for SAP HANA to RHEL 9 on ppc64le (#1042) - [IPU 8 -> 9] Improve the handling of blocklisted certificates (#992) ## Additional changes interesting for devels - Started work on bringing up networking inside the upgrade initramfs - currently available for testing and development purposes when LEAPP_DEVEL_INITRAM_NETWORK is set (#960) - Add leapp debug tools to the upgrade initramfs - dracut upgrade module (#997) - Enable disabling of DNF plugins in the dnfplugin library (#990)

evgeni force-pushed the bz2142270 branch from 81af404 to 8c014f1 Compare January 27, 2023 14:06

fernflower reviewed Jan 27, 2023

View reviewed changes

BZ#2142270 - reindex the database *before* the installer runs

70e6ce1

evgeni force-pushed the bz2142270 branch from 8c014f1 to 70e6ce1 Compare January 27, 2023 15:28

pirat89 requested changes Jan 27, 2023

View reviewed changes

pirat89 approved these changes Jan 30, 2023

View reviewed changes

pirat89 merged commit 896fa04 into oamg:master Jan 30, 2023

pirat89 added the changelog-checked The merger/reviewer checked the changelog draft document and updated it when relevant label Jan 30, 2023

pirat89 mentioned this pull request Feb 21, 2023

# Release 0.18.0 #1049

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

BZ#2142270 - reindex the database before the installer runs #1018

BZ#2142270 - reindex the database before the installer runs #1018

evgeni commented Jan 27, 2023

github-actions bot commented Jan 27, 2023

fernflower commented Jan 27, 2023

fernflower left a comment

fernflower Jan 27, 2023

evgeni Jan 27, 2023

pirat89 Jan 27, 2023 •

edited

fernflower commented Jan 27, 2023

github-actions bot commented Jan 27, 2023

github-actions bot commented Jan 27, 2023

github-actions bot commented Jan 27, 2023

github-actions bot commented Jan 27, 2023

github-actions bot commented Jan 27, 2023

github-actions bot commented Jan 27, 2023

github-actions bot commented Jan 27, 2023

pirat89 left a comment

lpramuk commented Jan 30, 2023

pirat89 left a comment

BZ#2142270 - reindex the database *before* the installer runs #1018

BZ#2142270 - reindex the database *before* the installer runs #1018

Conversation

evgeni commented Jan 27, 2023

github-actions bot commented Jan 27, 2023

Thank you for contributing to the Leapp project!

fernflower commented Jan 27, 2023

fernflower left a comment

Choose a reason for hiding this comment

fernflower Jan 27, 2023

Choose a reason for hiding this comment

evgeni Jan 27, 2023

Choose a reason for hiding this comment

pirat89 Jan 27, 2023 • edited

Choose a reason for hiding this comment

fernflower commented Jan 27, 2023

github-actions bot commented Jan 27, 2023

github-actions bot commented Jan 27, 2023

github-actions bot commented Jan 27, 2023

github-actions bot commented Jan 27, 2023

github-actions bot commented Jan 27, 2023

github-actions bot commented Jan 27, 2023

github-actions bot commented Jan 27, 2023

pirat89 left a comment

Choose a reason for hiding this comment

lpramuk commented Jan 30, 2023

pirat89 left a comment

Choose a reason for hiding this comment

BZ#2142270 - reindex the database before the installer runs #1018

BZ#2142270 - reindex the database before the installer runs #1018

pirat89 Jan 27, 2023 •

edited