Fixes #32488 - use rpm to remove pulp2 packages #473

jlsherrill · 2021-05-20T10:08:44Z

as yum can remove other packages unexpectedly
(such as mod_ssl and more)

theforeman-bot · 2021-05-20T10:08:53Z

jlsherrill · 2021-05-20T10:09:31Z

definitions/procedures/pulp/remove.rb

+                        python-pulp-manifest python-pulp-oid_validation python-pulp-ostree-common python-pulp-puppet-common
+                        python-pulp-python-common python-pulp-repoauth python-pulp-rpm-common python-pulp-streamer python-pymongo
+                        python-pymongo-gridfs python2-amqp python2-billiard python2-celery python2-debpkgr python2-django python2-kombu
+                        python2-solv python2-vine)


This is a list of every rpm in the latest pulp2 repository, minus gofer packages which can be used for katello-agent. No additional packages are listed

jlsherrill · 2021-05-20T10:10:08Z

definitions/procedures/pulp/remove.rb

+                        python2-solv python2-vine)
+
+      @installed_pulp_packages ||= all_possible.select { |pkg| find_package(pkg) }
+      @installed_pulp_packages


here we only consider packages actually installed, and cache to not have to recalculate again when this method is called a 2nd time.

ekohl · 2021-05-20T10:45:26Z

definitions/procedures/pulp/remove.rb

+      possible = %w[pulp-admin-client pulp-agent pulp-consumer-client pulp-deb-admin-extensions
+                    pulp-deb-plugins pulp-docker-admin-extensions pulp-docker-plugins
+                    pulp-nodes-admin-extensions pulp-nodes-child pulp-nodes-common
+                    pulp-nodes-consumer-extensions pulp-nodes-parent pulp-ostree-admin-extensions
+                    pulp-ostree-plugins pulp-puppet-admin-extensions
+                    pulp-puppet-consumer-extensions pulp-puppet-handlers pulp-puppet-plugins
+                    pulp-puppet-tools pulp-python-admin-extensions pulp-python-plugins
+                    pulp-rpm-admin-extensions pulp-rpm-consumer-extensions pulp-rpm-handlers
+                    pulp-rpm-plugins pulp-rpm-yumplugins pulp-selinux pulp-server python-bson


Can't you use rpm -qa | grep ^pulp- or will that also match pulp 3 packages?

I think both these suggestions would work, but to be honest i prefer this explicit list especially given the side effects we saw from users.

ekohl · 2021-05-20T10:46:47Z

definitions/procedures/pulp/remove.rb

+                    python-pulp-agent-lib python-pulp-bindings python-pulp-client-lib
+                    python-pulp-common python-pulp-deb-common python-pulp-devel
+                    python-pulp-docker-common python-pulp-integrity python-pulp-manifest
+                    python-pulp-oid_validation python-pulp-ostree-common python-pulp-puppet-common
+                    python-pulp-python-common python-pulp-repoauth python-pulp-rpm-common
+                    python-pulp-streamer python-pymongo python-pymongo-gridfs python2-amqp


Same here with ^python-pulp-

upadhyeammit · 2021-05-26T08:34:06Z

Hello @jlsherrill

Changes look good to me! Apart from these changes can you also modify data_dir_removal_cmds as,

    def data_dir_removal_cmds
      pulp_data_dirs.collect { |dir| "rm -rf #{dir}" if File.directory?(dir)}.compact
    end

Accordingly remove directory check from delete_pulp_data. This way we will only show the directories we will actually remove and avoid the split in delete_pulp_data.

Thank you

jlsherrill · 2021-06-02T18:17:39Z

updated!

upadhyeammit · 2021-06-08T08:41:21Z

updated!

Hello @jlsherrill can you remove if File.directory?(cmd.split[2]) from delete_pulp_data as we already have checked this in data_dir_removal_cmds. Thank you!

jlsherrill · 2021-06-08T18:20:46Z

updated!

jturel · 2021-06-11T17:54:27Z

Hit this while testing on a 4.0 box I upgrade from 3.18:

foreman-maintain content remove-pulp2
Running Remove Pulp2 and mongodb packages and data
================================================================================
Remove pulp2: 

WARNING: All pulp2 packages will be removed with the following commands:

# rpm -e pulp-deb-plugins  pulp-docker-plugins  pulp-puppet-plugins  pulp-puppet-tools  pulp-rpm-plugins  pulp-selinux  pulp-server  python-bson  python-crane  python-isodate  python-mongoengine  python-nectar  python-pulp-common  python-pulp-deb-common  python-pulp-docker-common  python-pulp-oid_validation  python-pulp-puppet-common  python-pulp-repoauth  python-pulp-rpm-common  python-pulp-streamer  python-pymongo  python-pymongo-gridfs  python2-amqp  python2-billiard  python2-celery  python2-debpkgr  python2-django  python2-kombu  python2-solv  python2-vine
# yum remove rh-mongodb34-*

All pulp2 data will be removed.

# rm -rf /var/lib/pulp/published
# rm -rf /var/lib/pulp/uploads
# rm -rf /var/lib/mongodb/

Do you want to proceed?, [y(yes), q(quit)] y
/ Removing pulp2 packages                                             [FAIL]    
Failed executing rpm -e pulp-deb-plugins  pulp-docker-plugins  pulp-puppet-plugins  pulp-puppet-tools  pulp-rpm-plugins  pulp-selinux  pulp-server  python-bson  python-crane  python-isodate  python-mongoengine  python-nectar  python-pulp-common  python-pulp-deb-common  python-pulp-docker-common  python-pulp-oid_validation  python-pulp-puppet-common  python-pulp-repoauth  python-pulp-rpm-common  python-pulp-streamer  python-pymongo  python-pymongo-gridfs  python2-amqp  python2-billiard  python2-celery  python2-debpkgr  python2-django  python2-kombu  python2-solv  python2-vine, exit status 30:
 error: Failed dependencies:
        pulp-server >= 2.8 is needed by (installed) pulp-katello-1.0.3-1.el7.noarch
        python-pulp-rpm-common is needed by (installed) pulp-katello-1.0.3-1.el7.noarch
--------------------------------------------------------------------------------
Scenario [Remove Pulp2 and mongodb packages and data] failed.

The following steps ended up in failing state:

  [pulp-remove]

Resolve the failed steps and rerun
the command. In case the failures are false positives,
use --whitelist="pulp-remove"

jturel · 2021-06-11T17:57:50Z

I added pulp-katello to the rpms to be removed and it succeeded.

Here's what's left over containing "pulp" or "mongo":

pulp-client-1.0-1.noarch
pulpcore-selinux-1.2.4-1.el7.x86_64
python3-mongoengine-0.20.0-1.el7.noarch
python3-pulp-2to3-migration-0.11.1-1.el7.noarch
python3-pulp-certguard-1.1.0-1.el7.noarch
python3-pulp-container-2.2.2-1.el7.noarch
python3-pulpcore-3.9.1-2.el7.noarch
python3-pulp-deb-2.9.2-1.el7.noarch
python3-pulp-file-1.5.0-1.el7.noarch
python3-pulp-rpm-3.10.0-1.el7.noarch
python3-pymongo-3.11.0-1.el7.x86_64
tfm-rubygem-pulp_2to3_migration_client-0.7.0-1.el7.noarch
tfm-rubygem-pulp_ansible_client-0.6.0-1.el7.noarch
tfm-rubygem-pulp_certguard_client-1.0.3-1.el7.noarch
tfm-rubygem-pulp_container_client-2.2.0-1.el7.noarch
tfm-rubygem-pulpcore_client-3.9.0-1.el7.noarch
tfm-rubygem-pulp_deb_client-2.9.1-1.el7.noarch
tfm-rubygem-pulp_file_client-1.5.0-1.el7.noarch
tfm-rubygem-pulp_rpm_client-3.10.0-1.el7.noarch
tfm-rubygem-smart_proxy_pulp-2.1.0-3.fm2_2.el7.noarch

The leftovers seem valid - but what about:

==============================================================================================================================================================================================
 Package                                                    Arch                                  Version                                      Repository                                Size
==============================================================================================================================================================================================
Removing:
 python3-pymongo                                            x86_64                                3.11.0-1.el7                                 @pulpcore                                3.4 M
Removing for dependencies:
 python3-mongoengine                                        noarch                                0.20.0-1.el7                                 @pulpcore                                1.0 M
 python3-pulp-2to3-migration                                noarch                                0.11.1-1.el7                                 @pulpcore                                1.1 M

as yum can remove other packages unexpectedly (such as mod_ssl and more)

jlsherrill · 2021-06-11T18:25:07Z

not sure why i didn't hit that, but easy enough to add! Thanks @jturel Added!

jturel

ACK - tests well!

jlsherrill · 2021-06-14T12:53:49Z

@upadhyeammit can you merge please :)

theforeman-bot added Needs testing Not yet reviewed labels May 20, 2021

jlsherrill commented May 20, 2021

View reviewed changes

jlsherrill force-pushed the 32488 branch from b3c3146 to 0bcf60e Compare May 20, 2021 10:20

ekohl reviewed May 20, 2021

View reviewed changes

jlsherrill force-pushed the 32488 branch from 0bcf60e to d298159 Compare June 2, 2021 18:12

jlsherrill force-pushed the 32488 branch from d298159 to efb5e39 Compare June 8, 2021 18:17

upadhyeammit approved these changes Jun 9, 2021

View reviewed changes

theforeman-bot removed the Not yet reviewed label Jun 9, 2021

Fixes #32488 - use rpm to remove pulp2 packages

81c3ac8

as yum can remove other packages unexpectedly (such as mod_ssl and more)

jlsherrill force-pushed the 32488 branch from efb5e39 to 81c3ac8 Compare June 11, 2021 18:24

jturel approved these changes Jun 11, 2021

View reviewed changes

upadhyeammit merged commit 073063b into theforeman:master Jun 14, 2021

jlsherrill deleted the 32488 branch June 15, 2021 14:42

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fixes #32488 - use rpm to remove pulp2 packages #473

Fixes #32488 - use rpm to remove pulp2 packages #473

jlsherrill commented May 20, 2021

theforeman-bot commented May 20, 2021

jlsherrill May 20, 2021

jlsherrill May 20, 2021

ekohl May 20, 2021

jlsherrill May 20, 2021

ekohl May 20, 2021

upadhyeammit commented May 26, 2021

jlsherrill commented Jun 2, 2021

upadhyeammit commented Jun 8, 2021

jlsherrill commented Jun 8, 2021

jturel commented Jun 11, 2021

jturel commented Jun 11, 2021 •

edited

jlsherrill commented Jun 11, 2021

jturel left a comment

jlsherrill commented Jun 14, 2021

Fixes #32488 - use rpm to remove pulp2 packages #473

Fixes #32488 - use rpm to remove pulp2 packages #473

Conversation

jlsherrill commented May 20, 2021

theforeman-bot commented May 20, 2021

jlsherrill May 20, 2021

Choose a reason for hiding this comment

jlsherrill May 20, 2021

Choose a reason for hiding this comment

ekohl May 20, 2021

Choose a reason for hiding this comment

jlsherrill May 20, 2021

Choose a reason for hiding this comment

ekohl May 20, 2021

Choose a reason for hiding this comment

upadhyeammit commented May 26, 2021

jlsherrill commented Jun 2, 2021

upadhyeammit commented Jun 8, 2021

jlsherrill commented Jun 8, 2021

jturel commented Jun 11, 2021

jturel commented Jun 11, 2021 • edited

jlsherrill commented Jun 11, 2021

jturel left a comment

Choose a reason for hiding this comment

jlsherrill commented Jun 14, 2021

jturel commented Jun 11, 2021 •

edited