Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes #27686, #27730, #27731, #27732 - Version locking NG #278

Merged
merged 2 commits into from Sep 5, 2019

Conversation

mbacovsky
Copy link
Member

Original locking of foreman-related packages was replaced
with new approach that locks all packages except some select
exceptions. For better performance and usability we are introducing
new yum plugin the will be used instead of the yum versionlock plugin.

  • the new yum plugin lives in extras/foreman_protector
  • the plugin excludes everything except for pkgs in whitelist
  • the plugin prints how many packages are excluded
  • the plugin prints hint about using f-m packages install/update
  • 'f-m packages install/update' install/update packages
    it unlocks them first, run the yum and run the installer --upgrade
    which locks packages again. User is informed and asked for confirmation
  • f-m packages status was extended to print if the locking is enabled
    If not, warning is printed.
  • packages commands can install and setup the new plugin

@theforeman-bot
Copy link
Member

Issues: #27686 #27730 #27731 #27732

1 similar comment
@theforeman-bot
Copy link
Member

Issues: #27686 #27730 #27731 #27732

@mbacovsky
Copy link
Member Author

@kgaikwad, @jameerpathan111, @upadhyeammit, could please take a look?

@upadhyeammit
Copy link
Contributor

upadhyeammit commented Aug 29, 2019

  1. Running is-locked first time on system,
# bin/foreman-maintain packages is-locked
Foreman related packages are not locked
  1. Locking the packages,
# bin/foreman-maintain packages lock
Running preparation steps required to run the next scenarios
================================================================================
Check if tooling for package locking is installed:                    [OK]
--------------------------------------------------------------------------------

Running locking of package versions
================================================================================
Lock versions of Foreman-related packages:                            [OK]
--------------------------------------------------------------------------------

# bin/foreman-maintain packages is-locked
Foreman related packages are locked
  1. Now checking the packages status, I think it should not allow to lock packages when Locking is not supported ?
# bin/foreman-maintain packages status
Running preparation steps required to run the next scenarios
================================================================================
Check if tooling for package locking is installed:                    [OK]
--------------------------------------------------------------------------------


Running detection of status of package version locking
================================================================================
Check status of version locking of packages: 
Locking of package versions is not supported.                         [OK]
--------------------------------------------------------------------------------
  1. Same with unlock ?
# bin/foreman-maintain packages unlock
Running preparation steps required to run the next scenarios
================================================================================
Check if tooling for package locking is installed:                    [OK]
--------------------------------------------------------------------------------
Running unlocking of package versions
================================================================================
Unlock versions of Foreman-related packages:                          [OK]
--------------------------------------------------------------------------------

@jameerpathan111
Copy link
Contributor

jameerpathan111 commented Aug 29, 2019

  • Error as "Config error: Couldn't parse /etc/yum/pluginconf.d/foreman-protector.conf: File contains no section headers." if answered no for preparation steps of tooling for package locking :
[root@qe-sat62-rhel7 foreman_maintain]# ./bin/foreman-maintain packages install foreman-discovery-image
Running preparation steps required to run the next scenarios
================================================================================
Check if tooling for package locking is installed:                    [FAIL]
Tools for package version locking are not available on this system
--------------------------------------------------------------------------------
Continue with step [Install and configure tools for version locking]?, [y(yes), n(no), q(quit)] n
Scenario [preparation steps required to run the next scenarios] failed.         

The following steps ended up in failing state:

  [version-locking-enabled]

Resolve the failed steps and rerun
the command. In case the failures are false positives,
use --whitelist="version-locking-enabled"
[root@qe-sat62-rhel7 foreman_maintain]# ./bin/foreman-maintain packages lock
Running preparation steps required to run the next scenarios
================================================================================
Check if tooling for package locking is installed:                    [OK]
--------------------------------------------------------------------------------


Running locking of package versions
================================================================================
Lock versions of Foreman-related packages:                            [OK]
--------------------------------------------------------------------------------
[root@qe-sat62-rhel7 foreman_maintain]# yum repolist 
Config error: Couldn't parse /etc/yum/pluginconf.d/foreman-protector.conf: File contains no section headers.
file: file:///etc/yum/pluginconf.d/foreman-protector.conf, line: 1
'enabled = 1\n'

  • Packages install do not respect assumeyes.
[root@qe-sat62-rhel7 foreman_maintain]# ./bin/foreman-maintain packages install foreman-discovery-image --assumeyes
Running preparation steps required to run the next scenarios
================================================================================
Check if tooling for package locking is installed:                    [OK]
--------------------------------------------------------------------------------


Running install packages in unlocked session
================================================================================
Confirm installer run is allowed: 
WARNING: This script runs satellite-installer after the yum execution 
to ensure the Satellite is in a consistent state.
As a result some of your services may be restarted. 

Do you want to proceed?, [y(yes), q(quit)] 
  • Should not go forward if answered no for "Continue with step [Install and configure tools for version locking]?"
[root@qe-sat62-rhel7 foreman_maintain]# ./bin/foreman-maintain packages install  foreman-discovery-image
Running preparation steps required to run the next scenarios
================================================================================
Check if tooling for package locking is installed:                    [FAIL]
Tools for package version locking are not available on this system
--------------------------------------------------------------------------------
Continue with step [Install and configure tools for version locking]?, [y(yes), n(no), q(quit)] n
Scenario [preparation steps required to run the next scenarios] failed.         

The following steps ended up in failing state:

  [version-locking-enabled]

Resolve the failed steps and rerun
the command. In case the failures are false positives,
use --whitelist="version-locking-enabled"



Running install packages in unlocked session
================================================================================
Confirm installer run is allowed: 
WARNING: This script runs satellite-installer after the yum execution 
to ensure the Satellite is in a consistent state.
As a result some of your services may be restarted. 

Do you want to proceed?, [y(yes), q(quit)] q
                                                                      [ABORTED] 
--------------------------------------------------------------------------------
Scenario [install packages in unlocked session] failed.

The processing was aborted by user during the following steps:

  [packages-installer-confirmation]
  • If packages are not installed or packages are unavailable, installer should not run.
[root@qe-sat62-rhel7 foreman_maintain]# ./bin/foreman-maintain packages install zsh,foreman-discovery-image
Running preparation steps required to run the next scenarios
================================================================================
Check if tooling for package locking is installed:                    [OK]
--------------------------------------------------------------------------------


Running install packages in unlocked session
================================================================================
Confirm installer run is allowed: 
WARNING: This script runs satellite-installer after the yum execution 
to ensure the Satellite is in a consistent state.
As a result some of your services may be restarted. 

Do you want to proceed?, [y(yes), q(quit)] y
                                                                      [OK]      
--------------------------------------------------------------------------------
Unlock versions of Foreman-related packages:                          [OK]
--------------------------------------------------------------------------------
Install packages: Error: Nothing to do
                                                    [WARNING]
Failed executing yum install zsh,foreman-discovery-image, exit status 1
--------------------------------------------------------------------------------
Running satellite-installer --upgrade --disable-system-checks: Upgrading, to monitor the progress on all related services, please do:
  foreman-tail | tee upgrade-$(date +%Y-%m-%d-%H%M).log
Upgrade Step: stop_services...
Running Stop Services

  • Unable to install dependencies of fio package if packages are locked.
# ./bin/foreman-maintain packages lock
Running preparation steps required to run the next scenarios
================================================================================
Check if tooling for package locking is installed: [OK]
--------------------------------------------------------------------------------

Running locking of package versions
================================================================================
Lock versions of Foreman-related packages: [OK]
--------------------------------------------------------------------------------
# ./bin/foreman-maintain upgrade run -y --target-version 6.6.z
Running preparation steps required to run the next scenarios
================================================================================
Install packages: Error: Package: fio-3.7-1.el7.x86_64 (rhel-7-server-rpms)
 Requires: libpmemblk.so.1()(64bit)
Error: Package: fio-3.7-1.el7.x86_64 (rhel-7-server-rpms)
 Requires: libpmemblk.so.1(LIBPMEMBLK_1.0)(64bit)
Error: Package: fio-3.7-1.el7.x86_64 (rhel-7-server-rpms)
 Requires: libpmem.so.1(LIBPMEM_1.0)(64bit)
Error: Package: fio-3.7-1.el7.x86_64 (rhel-7-server-rpms)
 Requires: libpmem.so.1()(64bit)
**********************************************************************
yum can be configured to try to resolve such errors by temporarily enabling
disabled repos and searching for missing dependencies.
To enable this functionality please set 'notify_only=0' in /etc/yum/pluginconf.d/search-disabled-repos.conf
**********************************************************************
Error: Package: fio-3.7-1.el7.x86_64 (rhel-7-server-rpms)
 Requires: libpmemblk.so.1()(64bit)
Error: Package: fio-3.7-1.el7.x86_64 (rhel-7-server-rpms)
 Requires: libpmemblk.so.1(LIBPMEMBLK_1.0)(64bit)
Error: Package: fio-3.7-1.el7.x86_64 (rhel-7-server-rpms)
 Requires: libpmem.so.1(LIBPMEM_1.0)(64bit)
Error: Package: fio-3.7-1.el7.x86_64 (rhel-7-server-rpms)
 Requires: libpmem.so.1()(64bit)
 You could try using --skip-broken to work around the problem
 You could try running: rpm -Va --nofiles --nodigest
 [FAIL]
Failed executing yum -y install fio, exit status 1
--------------------------------------------------------------------------------
Scenario [preparation steps required to run the next scenarios] failed.
The following steps ended up in failing state:
 
  • It should be up to user whether he wants to run satellite-installer after package update/install.
  • Change help message of packages command.
  • https://projects.theforeman.org/issues/27731 - not fixed (should show warning if packages are unlocked)(shouldn't depend on --[no-]lock-package-versions)

@jameerpathan111
Copy link
Contributor

1. Now checking the packages status, I think it should not allow to lock packages when Locking is not supported ?

+1, If we don't support it it then it would be better if we don't even lock packages for that Satellite version.

Copy link
Member

@kgaikwad kgaikwad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mbacovsky,
Sorry for the delay. Added just a small inline comments.
Apart from comments by @upadhyeammit and @jameerpathan111, no functionality wise comments from my side.

definitions/features/installer.rb Outdated Show resolved Hide resolved
definitions/procedures/packages/install.rb Outdated Show resolved Hide resolved
definitions/procedures/packages/locking_status.rb Outdated Show resolved Hide resolved
@mbacovsky
Copy link
Member Author

Thanks for the feedback!
@upadhyeammit all comments should be addressed.
@jameerpathan111 I have a few questions:
1/ Couldn't parse /etc/yum/pluginconf.d/foreman-protector.conf - I'm not able to reproduce this. If you skip the installation no plugin should be installed and yum has no reason to try to parse the config. Did you try to install manually? There is missing the '[main]' section and then the message is appropriate. Could you please provide more details how did you test this ad would be the expected outcome?
2/ For the install and --assumeyes - the format is packages install [options] <packages> so the -y should go before the packages and that works. Would you prefer to have rather option --packages instead of the package arguments?
3/ If packages are not installed installer should not run - I didn't found a way to detect if yum installed anything or not so the installer runs always. Suggestions welcome. If there is a time I'll try to patch this with some repoquery guess if there is anything to install and perhaps skip yum and installer at all.
4/ Skipping installer should not be supported scenario as it may be risky. There is workaround for the adventurous users - unlock, install and lock manually in three steps.
5/ Should not go forward if answered 'no' - I was not able to do that. May need fm internals change, investigating.

All other comments should be addressed.

Please give it another round.

@mbacovsky
Copy link
Member Author

@kgaikwad thanks for review. Updated, please re-check.

@mbacovsky
Copy link
Member Author

@upadhyeammit, @jameerpathan111 re 5/ I found out this is actually regression caused by fixing https://bugzilla.redhat.com/show_bug.cgi?id=1710305. I'm going to revert that fix and find better solution.

@jameerpathan111
Copy link
Contributor

@mbacovsky thanks for quickly working on issues. :)

@jameerpathan111 I have a few questions:
1/ Couldn't parse /etc/yum/pluginconf.d/foreman-protector.conf - I'm not able to reproduce this. If you skip the installation no plugin should be installed and yum has no reason to try to parse the config. Did you try to install manually? There is missing the '[main]' section and then the message is appropriate. Could you please provide more details how did you test this ad would be the expected outcome?

File packages_locking_sat63_testing.txt contains the steps I did to reproduce this issue.
I will again try to reproduce this on Satellite 6.6 tomorrow and give you exact steps.

2/ For the install and --assumeyes - the format is packages install [options] <packages> so the -y should go before the packages and that works. Would you prefer to have rather option --packages instead of the package arguments?

Ohh so that's how it was supposed to be used. It's ok then, we can keep it as it is.

3/ If packages are not installed installer should not run - I didn't found a way to detect if yum installed anything or not so the installer runs always. Suggestions welcome. If there is a time I'll try to patch this with some repoquery guess if there is anything to install and perhaps skip yum and installer at all.

Yeah, it would be great if we solve this.

4/ Skipping installer should not be supported scenario as it may be risky. There is workaround for the adventurous users - unlock, install and lock manually in three steps.

Yes, it makes sense to not skip installer.

@mbacovsky

  • Current behaviour of lock command is to lock everything including packages which are not installed. If we are not going to allow installation of available packages then we should change help
    message of packages command to something which describes this behaviour clearly.
    Currently it says Lock/Unlock installed packages
  • Also I am still getting dependency error while trying to install fio during upgrade run.
Error: Package: libpmemblk-1.5.1-2.1.el7.x86_64 (rhel-7-server-rpms)
           Requires: libdaxctl.so.1(LIBDAXCTL_2)(64bit)
Error: Package: libpmemblk-1.5.1-2.1.el7.x86_64 (rhel-7-server-rpms)
           Requires: libndctl.so.6()(64bit)
Error: Package: libpmemblk-1.5.1-2.1.el7.x86_64 (rhel-7-server-rpms)
           Requires: libndctl.so.6(LIBNDCTL_1)(64bit)
Error: Package: libpmemblk-1.5.1-2.1.el7.x86_64 (rhel-7-server-rpms)
           Requires: libndctl.so.6(LIBNDCTL_14)(64bit)
Error: Package: libpmemblk-1.5.1-2.1.el7.x86_64 (rhel-7-server-rpms)
           Requires: libndctl.so.6(LIBNDCTL_3)(64bit)
Error: Package: libpmemblk-1.5.1-2.1.el7.x86_64 (rhel-7-server-rpms)
           Requires: libdaxctl.so.1()(64bit)
**********************************************************************
yum can be configured to try to resolve such errors by temporarily enabling
disabled repos and searching for missing dependencies.
To enable this functionality please set 'notify_only=0' in /etc/yum/pluginconf.d/search-disabled-repos.conf
**********************************************************************

Error: Package: libpmemblk-1.5.1-2.1.el7.x86_64 (rhel-7-server-rpms)
           Requires: libdaxctl.so.1(LIBDAXCTL_2)(64bit)
Error: Package: libpmemblk-1.5.1-2.1.el7.x86_64 (rhel-7-server-rpms)
           Requires: libndctl.so.6()(64bit)
Error: Package: libpmemblk-1.5.1-2.1.el7.x86_64 (rhel-7-server-rpms)
           Requires: libndctl.so.6(LIBNDCTL_1)(64bit)
Error: Package: libpmemblk-1.5.1-2.1.el7.x86_64 (rhel-7-server-rpms)
           Requires: libndctl.so.6(LIBNDCTL_14)(64bit)
Error: Package: libpmemblk-1.5.1-2.1.el7.x86_64 (rhel-7-server-rpms)
           Requires: libndctl.so.6(LIBNDCTL_3)(64bit)
Error: Package: libpmemblk-1.5.1-2.1.el7.x86_64 (rhel-7-server-rpms)
           Requires: libdaxctl.so.1()(64bit)
 You could try using --skip-broken to work around the problem
 You could try running: rpm -Va --nofiles --nodigest
  • Even after adding name of package in extras/foreman_protector/foreman-protector.whitelist file, I am still unable to install that package using yum command.
[root@hpe-dl380egen8-01 foreman_maintain]# yum install zsh
Error: Nothing to do
  • This is how the output of status command is when autolocking is disabled in installer.
[root@hpe-dl380egen8-01 foreman_maintain]# satellite-installer --no-lock-package-versions
Package versions are locked. Continuing with unlock.
Installing             Done                                               [100%] [...........................................................................]
  Success!
  * Satellite is running at https://hostname.example.com

  * To install an additional Capsule on separate machine continue by running:

      capsule-certs-generate --foreman-proxy-fqdn "$CAPSULE" --certs-tar "/root/$CAPSULE-certs.tar"

  * To upgrade an existing 6.5 Capsule to 6.6:
      Please see official documentation for steps and parameters to use when upgrading a 6.5 Capsule to 6.6.

  The full log is at /var/log/foreman-installer/satellite.log
[root@hpe-dl380egen8-01 foreman_maintain]# ./bin/foreman-maintain packages status
Running preparation steps required to run the next scenarios
================================================================================
Check if tooling for package locking is installed:                    [OK]
--------------------------------------------------------------------------------


Running detection of status of package version locking
================================================================================
Check status of version locking of packages: 
  Automatic locking of package versions is disabled in installer.
  Packages are not locked.
  WARNING: When locking is disabled there is a risk of unwanted update
  of Satellite' and its components and possible data inconsistency    [OK]
--------------------------------------------------------------------------------

@mbacovsky
Copy link
Member Author

File packages_locking_sat63_testing.txt contains the steps I did to reproduce this issue.
I will again try to reproduce this on Satellite 6.6 tomorrow and give you exact steps.

Unfortunately it didn't help me to reproduce. Could you sent me content of the /etc/yum/pluginconf.d/foreman-protector.conf and check if it is supplied by the tests?

  • Current behaviour of lock command is to lock everything including packages which are not installed. If we are not going to allow installation of available packages then we should change help
    message of packages command to something which describes this behaviour clearly.
    Currently it says Lock/Unlock installed packages

Changed to Lock/Unlock package protection, install, update. is that better?

  • Also I am still getting dependency error while trying to install fio during upgrade run.
Error: Package: libpmemblk-1.5.1-2.1.el7.x86_64 (rhel-7-server-rpms)
           Requires: libdaxctl.so.1(LIBDAXCTL_2)(64bit)
Error: Package: libpmemblk-1.5.1-2.1.el7.x86_64 (rhel-7-server-rpms)
           Requires: libndctl.so.6()(64bit)
Error: Package: libpmemblk-1.5.1-2.1.el7.x86_64 (rhel-7-server-rpms)
           Requires: libndctl.so.6(LIBNDCTL_1)(64bit)
Error: Package: libpmemblk-1.5.1-2.1.el7.x86_64 (rhel-7-server-rpms)
           Requires: libndctl.so.6(LIBNDCTL_14)(64bit)
Error: Package: libpmemblk-1.5.1-2.1.el7.x86_64 (rhel-7-server-rpms)
           Requires: libndctl.so.6(LIBNDCTL_3)(64bit)
Error: Package: libpmemblk-1.5.1-2.1.el7.x86_64 (rhel-7-server-rpms)
           Requires: libdaxctl.so.1()(64bit)
**********************************************************************
yum can be configured to try to resolve such errors by temporarily enabling
disabled repos and searching for missing dependencies.
To enable this functionality please set 'notify_only=0' in /etc/yum/pluginconf.d/search-disabled-repos.conf
**********************************************************************

Error: Package: libpmemblk-1.5.1-2.1.el7.x86_64 (rhel-7-server-rpms)
           Requires: libdaxctl.so.1(LIBDAXCTL_2)(64bit)
Error: Package: libpmemblk-1.5.1-2.1.el7.x86_64 (rhel-7-server-rpms)
           Requires: libndctl.so.6()(64bit)
Error: Package: libpmemblk-1.5.1-2.1.el7.x86_64 (rhel-7-server-rpms)
           Requires: libndctl.so.6(LIBNDCTL_1)(64bit)
Error: Package: libpmemblk-1.5.1-2.1.el7.x86_64 (rhel-7-server-rpms)
           Requires: libndctl.so.6(LIBNDCTL_14)(64bit)
Error: Package: libpmemblk-1.5.1-2.1.el7.x86_64 (rhel-7-server-rpms)
           Requires: libndctl.so.6(LIBNDCTL_3)(64bit)
Error: Package: libpmemblk-1.5.1-2.1.el7.x86_64 (rhel-7-server-rpms)
           Requires: libdaxctl.so.1()(64bit)
 You could try using --skip-broken to work around the problem
 You could try running: rpm -Va --nofiles --nodigest

Fixed.

  • Even after adding name of package in extras/foreman_protector/foreman-protector.whitelist file, I am still unable to install that package using yum command.
[root@hpe-dl380egen8-01 foreman_maintain]# yum install zsh
Error: Nothing to do

The plugin reads the list from /etc/yum/pluginconf.d/foreman-protector.whitelist. If you add name of the package to the extras/ you need to remove the file from /etc and f-m re-installs it with version from extras. I'd recommend to edit the one in /etc directly.

  • This is how the output of status command is when autolocking is disabled in installer.
[root@hpe-dl380egen8-01 foreman_maintain]# satellite-installer --no-lock-package-versions
Package versions are locked. Continuing with unlock.
Installing             Done                                               [100%] [...........................................................................]
  Success!
  * Satellite is running at https://hostname.example.com

  * To install an additional Capsule on separate machine continue by running:

      capsule-certs-generate --foreman-proxy-fqdn "$CAPSULE" --certs-tar "/root/$CAPSULE-certs.tar"

  * To upgrade an existing 6.5 Capsule to 6.6:
      Please see official documentation for steps and parameters to use when upgrading a 6.5 Capsule to 6.6.

  The full log is at /var/log/foreman-installer/satellite.log
[root@hpe-dl380egen8-01 foreman_maintain]# ./bin/foreman-maintain packages status
Running preparation steps required to run the next scenarios
================================================================================
Check if tooling for package locking is installed:                    [OK]
--------------------------------------------------------------------------------


Running detection of status of package version locking
================================================================================
Check status of version locking of packages: 
  Automatic locking of package versions is disabled in installer.
  Packages are not locked.
  WARNING: When locking is disabled there is a risk of unwanted update
  of Satellite' and its components and possible data inconsistency    [OK]
--------------------------------------------------------------------------------

Correct, is it okay or any changes required?

@jameerpathan111
Copy link
Contributor

Unfortunately it didn't help me to reproduce. Could you sent me content of the /etc/yum/pluginconf.d/foreman-protector.conf and check if it is supplied by the tests?

Ok, I will send it.

Changed to Lock/Unlock package protection, install, update. is that better?

yes, it's better now.

  • Also I am still getting dependency error while trying to install fio during upgrade run.
    Fixed.

yeah, it seems to be fixed now.

The plugin reads the list from /etc/yum/pluginconf.d/foreman-protector.whitelist. If you add name of the package to the extras/ you need to remove the file from /etc and f-m re-installs it with version from extras. I'd recommend to edit the one in /etc directly.

Thanks for clarifying, it sure does work when I list package in whitelist file.

Correct, is it okay or any changes required?

It's good, now we have warning message in it.

@upadhyeammit
Copy link
Contributor

Hello,

Regarding

  1. 3/ If packages are not installed installer should not run - I didn't found a way to detect if yum installed anything or not so the installer runs always. Suggestions welcome. If there is a time I'll try to patch this with some repoquery guess if there is anything to install and perhaps skip yum and installer at all.

Before actually running the yum install or update command we can try querying if repository has the package for install or update,

# yum list available 'vsftpd'
Available Packages
vsftpd.x86_64                                              3.0.2-25.el7                                               rhel-7-server-rpms
# echo $?
0

For unavailable package I can see return code is 1,

# yum list available 'vsftp'
Error: No matching Packages to list
# echo $?
1

I can see its working with wildcard, as it is common use case with yum,

# yum list available 'vsftp*'
Available Packages
vsftpd.x86_64                                              3.0.2-25.el7                                               rhel-7-server-rpms
# echo $?
0

Same goes with updates,

# yum list updates 'vsftpd'
Error: No matching Packages to list
# echo $?
1
# yum downgrade vsftpd -y
Removed:
  vsftpd.x86_64 0:3.0.2-25.el7                                                                                                          

Installed:
  vsftpd.x86_64 0:3.0.2-22.el7                                                                                                          

# yum list updates 'vsftpd'
Updated Packages
vsftpd.x86_64                                              3.0.2-25.el7                                               rhel-7-server-rpms
# echo $?
0

If I give another thought to this then having package available for install or update does not mean that installation or update will be successful, and if we really want to be sure if package got installed or updated then we can make use of yum history, but this will run after 'yum install/update package-name',

# yum history
ID     | Command line             | Date and time    | Action(s)      | Altered
-------------------------------------------------------------------------------
    45 | downgrade vsftpd -y      | 2019-09-04 04:44 | Downgrade      |    1   
    44 | install vsftpd           | 2019-09-04 04:43 | Install        |    1   
    43 | remove vsftpd            | 2019-09-04 04:22 | Erase          |    1   

I can see that yum only adds transaction detail in history if transaction was successful, so maybe we can record the recent transaction number for package(s) and then check if it has been increased; this is simple one. Else we can even pull full transaction history of package and decide to run or not run installer again,

# yum history pkg-list vsftpd
ID     | Action(s)      | Package                                              
-------------------------------------------------------------------------------
    45 | Downgrade      | vsftpd-3.0.2-22.el7.x86_64                           
    45 | Downgraded     |        3.0.2-25.el7.x86_64                           
    44 | Install        | vsftpd-3.0.2-25.el7.x86_64              

@jameerpathan111
Copy link
Contributor

@mbacovsky @upadhyeammit
I am testing this PR on Capsule 6.6
I have tried all the available subcommands of packages command and all of them works as expected. I have also tried enabling feature through installer and it also worked as expected.
I have observed one issue though:

  1. On capsule we can't install fio because of dependencies issue(even though I don't think we need to care about fio not gettting installed on capsule). Similar to problem which we have faced on Satellite. So I wanted to know if it possible to allow dependencies of packages listed in whitelist file to be able to install freely? OR Maybe it's not a good idea to do this.
[root@qe-capsule-feature-rhel7 foreman_maintain]# foreman-maintain packages is-locked
Packages are locked
[root@qe-capsule-feature-rhel7 foreman_maintain]# yum install fio
Loaded plugins: enabled_repos_upload, foreman-protector, langpacks, package_upload, product-id, search-disabled-repos, subscription-manager
Default_Organization_Sat6Capsule7_capsule7                                                                                             | 2.5 kB  00:00:00     
Default_Organization_Sat6Maintain7_maintain7                                                                                           | 2.1 kB  00:00:00     
Default_Organization_Sat6Tools7_sat6tool7                                                                                              | 2.1 kB  00:00:00     
rhel-7-server-ansible-2-rpms                                                                                                           | 2.3 kB  00:00:00     
rhel-7-server-rpms                                                                                                                     | 2.0 kB  00:00:00     
rhel-server-rhscl-7-rpms                                                                                                               | 2.0 kB  00:00:00     
Excluding 11395 updates due to foreman-protector. 
Use foreman-maintain packages install/update <package> 
to safely install packages without restrictions.
Resolving Dependencies
--> Running transaction check
---> Package fio.x86_64 0:3.7-1.el7 will be installed
--> Processing Dependency: librdmacm.so.1(RDMACM_1.0)(64bit) for package: fio-3.7-1.el7.x86_64
--> Processing Dependency: libpmemblk.so.1(LIBPMEMBLK_1.0)(64bit) for package: fio-3.7-1.el7.x86_64
--> Processing Dependency: libpmem.so.1(LIBPMEM_1.0)(64bit) for package: fio-3.7-1.el7.x86_64
--> Processing Dependency: libibverbs.so.1(IBVERBS_1.1)(64bit) for package: fio-3.7-1.el7.x86_64
--> Processing Dependency: libibverbs.so.1(IBVERBS_1.0)(64bit) for package: fio-3.7-1.el7.x86_64
--> Processing Dependency: librdmacm.so.1()(64bit) for package: fio-3.7-1.el7.x86_64
--> Processing Dependency: librbd.so.1()(64bit) for package: fio-3.7-1.el7.x86_64
--> Processing Dependency: librados.so.2()(64bit) for package: fio-3.7-1.el7.x86_64
--> Processing Dependency: libpmemblk.so.1()(64bit) for package: fio-3.7-1.el7.x86_64
--> Processing Dependency: libpmem.so.1()(64bit) for package: fio-3.7-1.el7.x86_64
--> Processing Dependency: libibverbs.so.1()(64bit) for package: fio-3.7-1.el7.x86_64
--> Running transaction check
---> Package libibverbs.x86_64 0:22.1-3.el7 will be installed
--> Processing Dependency: rdma-core(x86-64) = 22.1-3.el7 for package: libibverbs-22.1-3.el7.x86_64
---> Package libpmem.x86_64 0:1.5.1-2.1.el7 will be installed
---> Package libpmemblk.x86_64 0:1.5.1-2.1.el7 will be installed
--> Processing Dependency: libndctl.so.6(LIBNDCTL_3)(64bit) for package: libpmemblk-1.5.1-2.1.el7.x86_64
--> Processing Dependency: libndctl.so.6(LIBNDCTL_14)(64bit) for package: libpmemblk-1.5.1-2.1.el7.x86_64
--> Processing Dependency: libndctl.so.6(LIBNDCTL_1)(64bit) for package: libpmemblk-1.5.1-2.1.el7.x86_64
--> Processing Dependency: libdaxctl.so.1(LIBDAXCTL_2)(64bit) for package: libpmemblk-1.5.1-2.1.el7.x86_64
--> Processing Dependency: libndctl.so.6()(64bit) for package: libpmemblk-1.5.1-2.1.el7.x86_64
--> Processing Dependency: libdaxctl.so.1()(64bit) for package: libpmemblk-1.5.1-2.1.el7.x86_64
---> Package librados2.x86_64 1:10.2.5-4.el7 will be installed
--> Processing Dependency: libboost_random-mt.so.1.53.0()(64bit) for package: 1:librados2-10.2.5-4.el7.x86_64
--> Processing Dependency: libboost_iostreams-mt.so.1.53.0()(64bit) for package: 1:librados2-10.2.5-4.el7.x86_64
---> Package librbd1.x86_64 1:10.2.5-4.el7 will be installed
--> Processing Dependency: libboost_random-mt.so.1.53.0()(64bit) for package: 1:librbd1-10.2.5-4.el7.x86_64
--> Processing Dependency: libboost_iostreams-mt.so.1.53.0()(64bit) for package: 1:librbd1-10.2.5-4.el7.x86_64
---> Package librdmacm.x86_64 0:22.1-3.el7 will be installed
--> Running transaction check
---> Package daxctl-libs.x86_64 0:64.1-2.el7 will be installed
---> Package librados2.x86_64 1:10.2.5-4.el7 will be installed
--> Processing Dependency: libboost_random-mt.so.1.53.0()(64bit) for package: 1:librados2-10.2.5-4.el7.x86_64
--> Processing Dependency: libboost_iostreams-mt.so.1.53.0()(64bit) for package: 1:librados2-10.2.5-4.el7.x86_64
---> Package librbd1.x86_64 1:10.2.5-4.el7 will be installed
--> Processing Dependency: libboost_random-mt.so.1.53.0()(64bit) for package: 1:librbd1-10.2.5-4.el7.x86_64
--> Processing Dependency: libboost_iostreams-mt.so.1.53.0()(64bit) for package: 1:librbd1-10.2.5-4.el7.x86_64
---> Package ndctl-libs.x86_64 0:64.1-2.el7 will be installed
---> Package rdma-core.x86_64 0:22.1-3.el7 will be installed
--> Finished Dependency Resolution
Error: Package: 1:librados2-10.2.5-4.el7.x86_64 (rhel-7-server-rpms)
           Requires: libboost_random-mt.so.1.53.0()(64bit)
Error: Package: 1:librbd1-10.2.5-4.el7.x86_64 (rhel-7-server-rpms)
           Requires: libboost_random-mt.so.1.53.0()(64bit)
Error: Package: 1:librados2-10.2.5-4.el7.x86_64 (rhel-7-server-rpms)
           Requires: libboost_iostreams-mt.so.1.53.0()(64bit)
Error: Package: 1:librbd1-10.2.5-4.el7.x86_64 (rhel-7-server-rpms)
           Requires: libboost_iostreams-mt.so.1.53.0()(64bit)
 You could try using --skip-broken to work around the problem
 You could try running: rpm -Va --nofiles --nodigest
Uploading Enabled Repositories Report
Loaded plugins: foreman-protector, langpacks, product-id, subscription-manager
  1. We are not supporting this feature on capsule so can we have some sort of message saying that user can use this feature but it's not supported on capsule. What you guys think?

@jameerpathan111
Copy link
Contributor

@mbacovsky @kgaikwad

  • I have tried Satellite upgrade( Satellite 6.3 -> 6.4 -> 6.5 -> 6.6 ) using this PR and found no issues so far. All the upgrades completed successfully.
  • I also interrupted Satellite 6.5 -> Satellite 6.6 upgrade and then reran it and it also completed successfully.
  • I have also tried all the subcommands of foreman-maintain packages commands on these satellite and they seemed to be working as expected.

@jameerpathan111
Copy link
Contributor

* the plugin prints how many packages are excluded

@mbacovsky Where can I see how many packages are excluded?

* 'f-m packages install/update' install/update packages
  it unlocks them first, run the yum and run the installer --upgrade
  which locks packages again. User is informed and asked for confirmation

Only when user have enabled lock-package-versions from installer, otherwise if user have disabled lock-package-versions and used FM to manually lock packages then 'f-m packages install/update' install/update packages where it'll first unlock packages and run the yum but it won't lock packages again. For this scenario it's user who has to lock packages back.

@mbacovsky
Copy link
Member Author

@upadhyeammit thanks for your input on yum operations. I'll take a look if it is something we can fit in. I need to test how it works if you install/update multiple packages at once. If we would need to test packages one by one it may have terrible performance. Also there is a complication with using patterns like 'install rubygem-*'. Perhaps we could leave it for separate PR.

@jameerpathan111, thanks for extensive testing! It is much appreciated.

  • re 1/ fio on capsule. I was thinking of including all deps too. The problem was the ones you are missing are deps of deps and there we get easily to glibc, selinux and a lot of others that are risky to update. Then I decided to manage manually.

  • re 2/ +1 adding the message may help

  • re excluded packages/ It is in the yum output when the plugin is executed. I'll improve the message so that it shine out more. Currently it is:

# yum update candlepin                                                                                                                                                                
Loaded plugins: enabled_repos_upload, foreman-protector, package_upload, product-id, search-disabled-repos, subscription-manager
...   
Excluding 28689 updates due to foreman-protector. 
Use foreman-maintain packages install/update <package> 
to safely install packages without restrictions.
No packages marked for update
Uploading Enabled Repositories Report
Loaded plugins: foreman-protector, product-id, subscription-manager
  • re lock after install/ I think that if automatic locking is disabled it is okay if don't autolock packages after install. But I probably see your point - would it help if we run locking status after the installer so that it is visible in what state the locking is?

One last thing is the confirmation issue. Now with this patch we have back the double confirm message. I finally have patch and will include it in this PR as separate commit. Will update soon.

@jameerpathan111
Copy link
Contributor

* re 1/ fio on capsule. I was thinking of including all deps too. The problem was the ones you are missing are deps of deps and there we get easily to glibc, selinux and a lot of others that are risky to update.  Then I decided to manage manually.

yeah, it sure is risky to allow installing all dependencies.

* re 2/ +1 adding the message may help

+1

# yum update candlepin                                                                                                                                                                
Loaded plugins: enabled_repos_upload, foreman-protector, package_upload, product-id, search-disabled-repos, subscription-manager
...   
Excluding 28689 updates due to foreman-protector. 
Use foreman-maintain packages install/update <package> 
to safely install packages without restrictions.
No packages marked for update
Uploading Enabled Repositories Report
Loaded plugins: foreman-protector, product-id, subscription-manager

Martin I haven't seen this message on my Satellite so far. I had packages locked and ran yum install pkg-name, the only message I got was Error: Nothing to do.

* re lock after install/ I think that if automatic locking is disabled it is okay if don't autolock packages after install. But I probably see your point - would it help if we run locking status after the installer so that it is visible in what state the locking is?

Yeah, it'll be good to have such message.

One last thing is the confirmation issue. Now with this patch we have back the double confirm message. I finally have patch and will include it in this PR as separate commit. Will update soon.

Awsome

@mbacovsky
Copy link
Member Author

Updated.
@jameerpathan111,

  • updated the fio deps with the ones you are missing
  • the yum plugin message is better visible now. Is it really missing on your satellite? I can see it in your copy-paste from capsule.
  • lock status after yum install/update

TODO: message about unsupported locking on capsule.
Anything else is missing?

I included fix for RM#27072 for easier testing, it should go out in 0.4.7 to avoid the new regression introduced by 0.4.6. @kgaikwad, @upadhyeammit could you please take a look?

@jameerpathan111
Copy link
Contributor

* the yum plugin message is better visible now. Is it really missing on your satellite? I can see it in your copy-paste from capsule.

Yes, I can see yum plugin message on Capsule but not on Satellite.

@jameerpathan111
Copy link
Contributor

@mbacovsky

  • Now yum plugin is showing proper message on satellite.
[root@qe-sat62-rhel7 foreman_maintain]# yum install zsh
*** Excluded total: 11576
Excluding 11576 updates due to foreman-protector. 
Use foreman-maintain packages install/update <package> 
to safely install packages without restrictions.
Error: Nothing to do
  • About asking for permission to migrate to next sat version.
    Procedure before migrating to next sat version used to ask for permission twice.
Continue with [Procedures before migrating to Satellite 6.6], [y(yes), n(no), q(quit)] y
Continue with [Procedures before migrating to Satellite 6.6], [y(yes), n(no), q(quit)] y
Running Procedures before migrating to Satellite 6.6                            
================================================================================
disable active sync plans: 
\ Total 0 sync plans are now disabled.                                [OK]      
--------------------------------------------------------------------------------

Now the issue is fixed, this steps output is changed a little and it doesn't ask for confirmation twice.

The pre-upgrade checks indicate that the system is ready for upgrade.
It's recommended to perform a backup at this stage.
Confirm to continue with the modification part of the upgrade, [y(yes), n(no), q(quit)] y
Running Procedures before migrating to Satellite 6.6.z                          
================================================================================
disable active sync plans: 
/ Total 0 sync plans are now disabled.                                [OK]      
--------------------------------------------------------------------------------

Copy link
Contributor

@upadhyeammit upadhyeammit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked these changes couple of times and I dont see any modifications. Its handled in better way than before.

Copy link
Member

@kgaikwad kgaikwad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Big thank you @mbacovsky. Almost all review comments are addressed 🎉
Different scenarios tested by @jameerpathan111 already.

No any additional comment from my side.
Thanks everybody involved! Let's merge this PR :-)

Original locking of foreman-related packages was replaced
with new approach that locks all packages except some select
exceptions. For better performance and usability we are introducing
new yum plugin the will be used instead of the yum versionlock plugin.

- the new yum plugin lives in extras/foreman_protector
- the plugin excludes everything except for pkgs in whitelist
- the plugin prints how many packages are excluded
- the plugin prints hint about using f-m packages install/update
- 'f-m packages install/update' install/update packages
  it unlocks them first, run the yum and run the installer --upgrade
  which locks packages again. User is informed and asked for confirmation
- f-m packages status was extended to print if the locking is enabled
  If not, warning is printed.
- packages commands can install and setup the new plugin
@mbacovsky
Copy link
Member Author

Thank you all for your help!
@kgaikwad the manually squashed PRs were pushed. Tests are green. It is ready to merge.

@kgaikwad kgaikwad merged commit 7c070f7 into theforeman:master Sep 5, 2019
@kgaikwad
Copy link
Member

kgaikwad commented Sep 5, 2019

Thank you @mbacovsky!

@jameerpathan111
Copy link
Contributor

@mbacovsky is it possible to handle following scenario as well? :(

[root@fedora-build02 foreman_maintain]# foreman-maintain packages status
Running preparation steps required to run the next scenarios
================================================================================
Check if tooling for package locking is installed:                    [OK]
--------------------------------------------------------------------------------


Running detection of status of package version locking
================================================================================
Check status of version locking of packages: 
  Automatic locking of package versions is enabled in installer.
  Packages are locked.                                                [OK]
--------------------------------------------------------------------------------

[root@fedora-build02 foreman_maintain]# yum install https://dl.fedoraproject.org/pub/epel/7/x86_64/Packages/n/nss-mdns-0.14.1-1.el7.x86_64.rpm 

==============================================================================================================================================================
 Package                         Arch                          Version                             Repository                                            Size
==============================================================================================================================================================
Installing:
 nss-mdns                        x86_64                        0.14.1-1.el7                        /nss-mdns-0.14.1-1.el7.x86_64                        131 k

Transaction Summary
==============================================================================================================================================================
Install  1 Package

Total size: 131 k
Installed size: 131 k
Is this ok [y/d/N]: y

Installed:
  nss-mdns.x86_64 0:0.14.1-1.el7 

@mbacovsky
Copy link
Member Author

@jameerpathan111 that may be difficult to implement as it seems the local installs are handled differently by yum and our hook is ignored. Could you please create BZ to track this? We may give it a try later.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants