(PE-10915) Add SLES 10 upgrade for PE #63

jpartlow · 2015-09-15T23:17:41Z

Introduces support for upgrading SLES 10 agents in PE. SLES 10 does not
have tools for securely handling remote package management.

puppet-agent package is downloaded from a pe_packages fileserver mount
prepared separately on the master (in puppet_enterprise module)
the package is downloaded with a File resource, but only if agent node
is not already at the current aio version
because of limitations in the rpm package provider, it is not feasible
to get the install of the new puppet-agent package to run with '-U
--replacepkgs' flags that would allow rpm to correctly uninstall
conflicting superseded pe-* packages, so we remove those prior to
install. (Passing '-U --replacepkgs' in install_options parameter ends
up failing:

Error: Execution of '/bin/rpm -i "-U --replacepkgs"
/opt/puppetlabs/packages/puppet-agent-1.2.5-1.sles10.x86_64.rpm'
returned 1: -U --replacepkgs: not an rpm package (or package manifest)

I'm threading package_file_name parameter through because it varies by
platform and ends up being needed both in the prepare and the install
phase for these special case platforms.

I wasn't seeing expected failures when I started making SLES 10 available, so I switched to using expect { block }.to raise_error(/foo/) in order to verify that the tests were noticing that errors were no longer being thrown.

ericwilliamson · 2015-09-18T21:32:21Z

manifests/install.pp

+
+      contain puppet_agent::install::remove_packages
+
+      exec { 'replace puppet.conf removed by package removal':


Why do we need to uninstall everything? SLES 10 supports upgrading and dependency resolution.

b1o3ocw0433l686:~ # puppet --version 3.8.1 (Puppet Enterprise 3.8.2)

b1o3ocw0433l686:~ # rpm -qa | grep pe-* openssl-certs-0.8.0-0.8.29 perl-Carp-Clan-5.3-13.5 perl-Digest-SHA1-2.10-15.2 perl-URI-1.35-15.2 limal-nfs-server-perl-1.1.72-0.3 openslp-server-1.2.0-22.34.1 perl-X500-DN-0.28-133.2 perl-Date-Calc-5.4-14.5 pe-augeas-1.3.0-1.pe.sles10 pe-mcollective-common-2.7.0.1-1.pe.sles10 pe-ruby-rgen-0.6.5-3.pe.sles10 pe-mcollective-2.7.0.1-1.pe.sles10 compat-openssl097g-0.9.7g-13.19.1 libjpeg-6.2.0-752.2 device-mapper-1.02.13-6.18.7 openct-0.6.6-16.4.1 perl-5.8.8-14.17.15 limal-perl-1.1.72-0.3 openssl-0.9.8a-18.48.17 perl-Compress-Zlib-1.35-14.2 perl-Config-Crontab-1.11-12.2 perl-Digest-MD4-1.5-13.2 perl-Parse-RecDescent-1.80-259.2 perl-TimeDate-1.16-136.2 perl-XML-Parser-2.34-43.2 perl-gettext-1.05-13.2 openslp-1.2.0-22.34.1 perl-Crypt-SmbHash-0.12-13.2 permissions-2007.2.15-0.7 limal-ca-mgm-perl-1.1.75-0.3 perl-Bootloader-0.4.19.24-0.6.33 suspend-20071205-0.11.13 OpenIPMI-2.0.14-2.7.12 yast2-perl-bindings-2.13.11-0.22 zypper-0.6.201-0.4.1 pe-puppet-enterprise-release-3.8.2.0-1.pe.sles10 pe-libldap-2.4.39-5.pe.sles10 pe-libyaml-0.1.6-5.sles10 pe-ruby-stomp-1.3.3-1.pe.sles10 pe-ruby-augeas-0.5.0-7.pe.sles10 pe-hiera-1.3.4.5-1.pe.sles10 pe-ruby-shadow-2.2.0-4.pe.sles10 pe-facter-2.4.4.0-1.pe.sles10 pe-puppet-3.8.1.1.50-1.pe.sles10 freetype2-2.1.10-18.23.1 perl-Config-IniFiles-2.39-13.4 perl-TermReadKey-2.30-13.2 perl-XML-Writer-0.600-13.2 openldap2-client-2.3.32-0.37.1 perl-Bit-Vector-6.4-13.5 opensc-0.9.6-17.12 openssh-5.1p1-41.8.20 pe-openssl-1.0.0s-1.pe.sles10 pe-ruby-1.9.3.551-2.pe.sles10 pe-rubygem-deep-merge-1.0.0-6.pe.sles10 pe-virt-what-1.14-2.sles10 pe-ruby-ldap-0.9.12-7.pe.sles10

b1o3ocw0433l686:~ # md5sum /etc/puppetlabs/puppet/puppet.conf cab5c77b74732923b5830b9f4a1bdcab /etc/puppetlabs/puppet/puppet.conf b1o3ocw0433l686:~ # cat !$ cat /etc/puppetlabs/puppet/puppet.conf [main] vardir = /var/opt/lib/pe-puppet logdir = /var/log/pe-puppet rundir = /var/run/pe-puppet basemodulepath = /etc/puppetlabs/puppet/modules:/opt/puppet/share/puppet/modules user = pe-puppet group = pe-puppet archive_files = true server = pe38-latest [agent] report = true classfile = $vardir/classes.txt localconfig = $vardir/localconfig graph = true pluginsync = true certname = b1o3ocw0433l686.delivery.puppetlabs.net

b1o3ocw0433l686:~ # rpm -U --replacepkgs puppet-agent-1.2.5.rpm warning: puppet-agent-1.2.5.rpm: Header V3 RSA/SHA1 signature: NOKEY, key ID 4bd6ec30 warning: /etc/puppetlabs/mcollective/server.cfg created as /etc/puppetlabs/mcollective/server.cfg.rpmnew warning: /etc/puppetlabs/puppet/puppet.conf created as /etc/puppetlabs/puppet/puppet.conf.rpmnew insserv: script puppet: service puppet already provided! insserv: script pe-mcollective: service mcollective already provided! pe-puppet 0:off 1:off 2:off 3:off 4:on 5:off 6:off pe-mcollective 0:off 1:off 2:on 3:on 4:on 5:on 6:off

b1o3ocw0433l686:~ # /opt/puppetlabs/bin/puppet --version 4.2.2

b1o3ocw0433l686:~ # rpm -qa | grep pe-* openssl-certs-0.8.0-0.8.29 perl-Carp-Clan-5.3-13.5 perl-Digest-SHA1-2.10-15.2 perl-URI-1.35-15.2 limal-nfs-server-perl-1.1.72-0.3 openslp-server-1.2.0-22.34.1 perl-X500-DN-0.28-133.2 perl-Date-Calc-5.4-14.5 compat-openssl097g-0.9.7g-13.19.1 libjpeg-6.2.0-752.2 device-mapper-1.02.13-6.18.7 openct-0.6.6-16.4.1 perl-5.8.8-14.17.15 limal-perl-1.1.72-0.3 openssl-0.9.8a-18.48.17 perl-Compress-Zlib-1.35-14.2 perl-Config-Crontab-1.11-12.2 perl-Digest-MD4-1.5-13.2 perl-Parse-RecDescent-1.80-259.2 perl-TimeDate-1.16-136.2 perl-XML-Parser-2.34-43.2 perl-gettext-1.05-13.2 openslp-1.2.0-22.34.1 perl-Crypt-SmbHash-0.12-13.2 permissions-2007.2.15-0.7 limal-ca-mgm-perl-1.1.75-0.3 perl-Bootloader-0.4.19.24-0.6.33 suspend-20071205-0.11.13 OpenIPMI-2.0.14-2.7.12 yast2-perl-bindings-2.13.11-0.22 zypper-0.6.201-0.4.1 puppet-agent-1.2.5-1.sles10 freetype2-2.1.10-18.23.1 perl-Config-IniFiles-2.39-13.4 perl-TermReadKey-2.30-13.2 perl-XML-Writer-0.600-13.2 openldap2-client-2.3.32-0.37.1 perl-Bit-Vector-6.4-13.5 opensc-0.9.6-17.12 openssh-5.1p1-41.8.20

b1o3ocw0433l686:~ # md5sum /etc/puppetlabs/puppet/puppet.conf cab5c77b74732923b5830b9f4a1bdcab /etc/puppetlabs/puppet/puppet.conf b1o3ocw0433l686:~ # cat /etc/puppetlabs/puppet/puppet.conf [main] vardir = /var/opt/lib/pe-puppet logdir = /var/log/pe-puppet rundir = /var/run/pe-puppet basemodulepath = /etc/puppetlabs/puppet/modules:/opt/puppet/share/puppet/modules user = pe-puppet group = pe-puppet archive_files = true server = pe38-latest [agent] report = true classfile = $vardir/classes.txt localconfig = $vardir/localconfig graph = true pluginsync = true certname = b1o3ocw0433l686.delivery.puppetlabs.net

Didn't know rpm did that, I incorrectly assumed it was a yum feature when I first hit the failure; that's helpful. Is there a similar facility in Solaris 10/OSX package management? And do you happen to know if the aix rpm flavors would have the same dependency resolution?

Solaris and AIX most likely - that would be @haus or @geoffnichols

OSX - it does quasi upgrades. It will replace the package and all that good stuff, but it won't preserve any files. It will always over write them. However our puppet-agent package does that work for us of backing up puppet.conf and restoring it. see https://tickets.puppetlabs.com/browse/RE-5089

@ericwilliamson but will OSX remove the dependencies of the package being replaced?

Solaris has no such facility. uninstall and reinstall. solaris 11 supports upgrades, but this isn't a standard upgrade. solaris 10 doesn't even have upgrade, it is uninstall and reinstall.

@ericwilliamson what error do you get when you omit oldpackage?

buf6kr6un4xy104:~ # rpm -i -U --replacepkgs puppet-agent-1.2.5-1.sles10.x86_64.rpm warning: puppet-agent-1.2.5-1.sles10.x86_64.rpm: Header V3 RSA/SHA1 signature: NOKEY, key ID 4bd6ec30 warning: /etc/puppetlabs/puppet/puppet.conf created as /etc/puppetlabs/puppet/puppet.conf.rpmnew insserv: script puppet: service puppet already provided! insserv: script pe-mcollective: service mcollective already provided! pe-puppet 0:off 1:off 2:off 3:off 4:on 5:off 6:off pe-mcollective 0:off 1:off 2:on 3:on 4:on 5:on 6:off

okay, adding -U and --replacepkgs seems sane to me.

I thought I had explicitly tried that earlier; but I'll check it again.

Yeah:

Error: Execution of '/bin/rpm -i "-U --replacepkgs" /opt/puppetlabs/packages/puppet-agent-1.2.5-1.sles10.x86_64.rpm' returned 1: -U --replacepkgs: not an rpm package (or package manifest):
Error: /Stage[main]/Puppet_agent::Install/Package[puppet-agent]/ensure: change from absent to present failed: Execution of '/bin/rpm -i "-U --replacepkgs" /opt/puppetlabs/packages/puppet-agent-1.2.5-1.sles10.x86_64.rpm' returned 1: -U --replacepkgs: not an rpm package (or package manifest):

We were determining the master's agent_version based on file('/opt/puppetlabs/puppet/VERSION'). Since we need to refer to this in more than one class now, and since it is only used in PE, it makes sense to move it to the params class and to base it off of the pe_compiling_server_aio_build() function which inspects the same file. This way we're not duplicating the function across modules, and we're not mocking the builtin file() function specifically in our specs.

jpartlow · 2015-09-23T15:19:12Z

I've added specs and rebased.

Oops, Ruby 1.8.7 up with which will not put my chaining syntax.

ericwilliamson · 2015-09-23T16:36:04Z

manifests/install/remove_packages.pp

+# to installing puppet-agent.
+#
+class puppet_agent::install::remove_packages {
+  if versioncmp("${::clientversion}", '4.0.0') < 0 {


For consistency, should this class also contain an assert_private()?

Probably; that and prepare::packages. I'll add and fix up the spec syntax.

jpartlow · 2015-09-23T17:58:13Z

Finally. Specs passed.

jpartlow · 2015-09-23T17:59:01Z

We should probably hold of on merging this until we're satisfied with it, and the Solaris11 PR. I'm adding specs for that now

Introduces support for upgrading SLES 10 agents in PE. SLES 10 does not have tools for securely handling remote package management. * puppet-agent package is downloaded from a pe_packages fileserver mount prepared separately on the master (in puppet_enterprise module) * the package is downloaded with a File resource, but only if agent node is not already at the current aio version * because of limitations in the rpm package provider, it is not feasible to get the install of the new puppet-agent package to run with '-U --replacepkgs' flags that would allow rpm to correctly uninstall conflicting superseded pe-* packages, so we remove those prior to install. (Passing '-U --replacepkgs' in install_options parameter ends up failing: Error: Execution of '/bin/rpm -i "-U --replacepkgs" /opt/puppetlabs/packages/puppet-agent-1.2.5-1.sles10.x86_64.rpm' returned 1: -U --replacepkgs: not an rpm package (or package manifest) * I'm threading package_file_name parameter through because it varies by platform and ends up being needed both in the prepare and the install phase for these special case platforms.

...to indicate support for 10 and 11. I'm not entirely certain how rspec-puppet-facts works, but this then produced failures for SLES 11. SLES 10 and 12 aren't tested because rspec-puppet-facts doesn't seem to generate facts for them. The fix for SLES 11 was to set is_pe and operatingsystemmajrelease specifically, and to mock the pe only functions that the SLES classes need.

highb · 2015-09-23T22:56:51Z

manifests/init.pp

-      class { '::puppet_agent::install': } ->
+
+      if $::operatingsystem == 'SLES' and $::operatingsystemmajrelease == '10' {
+        $_package_file_name = "${puppet_agent::package_name}-${puppet_agent::params::master_agent_version}-1.sles10.${::architecture}.rpm"


Shouldn't these platform specific things be relegated off to params.pp?

Yes, we could do that, pull this into params as puppet_agent::params::package_file_name which defaults to undef, and then have puppet_agent::prepare::package_file_name and puppet_agent::install::package_file_name default to puppet_agent::params::package_file_name.

@highb I looked into this, but that name is dependent on puppet_agent::package_name which, given that it is a top level param could theoretically change, though I don't know why it would. Given the current structure seems best to keep it as is.

jpartlow · 2015-09-25T00:47:03Z

Manually tested an upgrade of Sles10 3.8.2 -> 2015.2.x with this patch.

highb · 2015-09-28T22:49:05Z

I've also tested an upgrade of Sles10 3.8.2 -> 2015.2.1 with this patch. Specs run fine. Let's ship this thing.

(PE-10915) Add SLES 10 upgrade for PE

(maint) Use rspec expect syntax for catching errors

588e103

I wasn't seeing expected failures when I started making SLES 10 available, so I switched to using expect { block }.to raise_error(/foo/) in order to verify that the tests were noticing that errors were no longer being thrown.

jpartlow force-pushed the issue/master/pe-10915-add-sles-10 branch from c8c91e1 to 86696fa Compare September 15, 2015 23:18

ericwilliamson reviewed Sep 18, 2015
View reviewed changes

jpartlow mentioned this pull request Sep 21, 2015

(PE-12001) Add solaris 10 #65

Merged

jpartlow force-pushed the issue/master/pe-10915-add-sles-10 branch from d3cd292 to 8b25a37 Compare September 23, 2015 14:38

jpartlow changed the title ~~WIP (PE-10915) Add sles 10~~ (PE-10915) Add SLES 10 upgrade for PE Sep 23, 2015

jpartlow force-pushed the issue/master/pe-10915-add-sles-10 branch 2 times, most recently from 7e08a47 to 8aa498b Compare September 23, 2015 15:18

ericwilliamson reviewed Sep 23, 2015
View reviewed changes

jpartlow force-pushed the issue/master/pe-10915-add-sles-10 branch from 8aa498b to 75e9923 Compare September 23, 2015 17:48

jpartlow force-pushed the issue/master/pe-10915-add-sles-10 branch from 75e9923 to 931bb66 Compare September 23, 2015 18:59

highb reviewed Sep 23, 2015
View reviewed changes

highb added a commit that referenced this pull request Sep 28, 2015

Merge pull request #63 from jpartlow/issue/master/pe-10915-add-sles-10

ab14bed

(PE-10915) Add SLES 10 upgrade for PE

highb merged commit ab14bed into puppetlabs:master Sep 28, 2015

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

(PE-10915) Add SLES 10 upgrade for PE #63

(PE-10915) Add SLES 10 upgrade for PE #63

jpartlow commented Sep 15, 2015

ericwilliamson Sep 18, 2015

jpartlow Sep 18, 2015

ericwilliamson Sep 18, 2015

jpartlow Sep 18, 2015

haus Sep 18, 2015

haus Sep 21, 2015

ericwilliamson Sep 21, 2015

haus Sep 21, 2015

haus Sep 21, 2015

jpartlow Sep 21, 2015

jpartlow commented Sep 23, 2015

ericwilliamson Sep 23, 2015

jpartlow Sep 23, 2015

jpartlow commented Sep 23, 2015

jpartlow commented Sep 23, 2015

highb Sep 23, 2015

jpartlow Sep 24, 2015

jpartlow Sep 24, 2015

jpartlow commented Sep 25, 2015

highb commented Sep 28, 2015


		contain puppet_agent::install::remove_packages

		exec { 'replace puppet.conf removed by package removal':

(PE-10915) Add SLES 10 upgrade for PE #63

(PE-10915) Add SLES 10 upgrade for PE #63

Conversation

jpartlow commented Sep 15, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jpartlow commented Sep 23, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jpartlow commented Sep 23, 2015

jpartlow commented Sep 23, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jpartlow commented Sep 25, 2015

highb commented Sep 28, 2015