Fixes #29190 - Support EL8 #77

wbclark · 2020-03-10T23:55:58Z

After marking el8 as a supported OS in metadata.json, I can run spec tests using el8 facts:

[wclark@perseids puppet-pulpcore]$ SPEC_FACTS_OS=centos-8-x86_64 bundle exec rspec spec/classes/plugin_migration_spec.rb 
...

Finished in 4.83 seconds (files took 5.29 seconds to load)
3 examples, 0 failures

[wclark@perseids puppet-pulpcore]$ SPEC_FACTS_OS=centos-8-x86_64 bundle exec rspec spec/classes/plugin_file_spec.rb 
...

Finished in 4.03 seconds (files took 5.54 seconds to load)
3 examples, 0 failures

[wclark@perseids puppet-pulpcore]$ SPEC_FACTS_OS=centos-8-x86_64 bundle exec rspec spec/classes/plugin_container_spec.rb 
...

Finished in 3.8 seconds (files took 5.1 seconds to load)
3 examples, 0 failures

[wclark@perseids puppet-pulpcore]$ SPEC_FACTS_OS=centos-8-x86_64 bundle exec rspec spec/classes/pulpcore_spec.rb 
.......

Finished in 19.54 seconds (files took 5.41 seconds to load)
7 examples, 0 failures

[wclark@perseids puppet-pulpcore]$

ekohl

In general you only do this if it's really supported. We have acceptance tests so please make sure they also run on EL8.

wbclark · 2020-03-12T18:45:00Z

In general you only do this if it's really supported. We have acceptance tests so please make sure they also run on EL8.

Currently the acceptance tests are blocked because pulpcore-selinux is not available in any el8 repos.

With that said, I obtained the package from http://koji.katello.org/koji/taskinfo?taskID=295341 and manually installed it on my test el8 system.

Once the package is installed, all acceptance tests are successful.

ekohl · 2020-03-12T19:17:59Z

theforeman/foreman-infra#1290 is open for that. IMHO we shouldn't merge partial support.

ehelms · 2020-03-13T14:55:09Z

.travis.yml would need EL8 added to it so that the EL8 versions of the acceptance run, the pulpcore-selinux issue has been fixed

http://koji.katello.org/releases/yum/katello-nightly/pulpcore/el8/x86_64/pulpcore-selinux-1.0.0-2.el8.x86_64.rpm

wbclark · 2020-03-13T16:59:00Z

centos8-64-1 12:54:17$ puppet apply --verbose --detailed-exitcodes /tmp/apply_manifest.pp.dsxZsX
  Info: Loading facts
  Info: Loading facts
  Info: Loading facts
  Info: Loading facts
  Info: Loading facts
  Info: Loading facts
  Info: Loading facts
  Notice: Compiled catalog for centos8-64-1 in environment production in 2.93 seconds
  Info: Applying configuration version '1584118461'
  Notice: /Stage[main]/Pulpcore::Config/File[/var/lib/pulp/tmp]/seltype: seltype changed 'var_lib_t' to 'pulpcore_var_lib_t'
  Info: Class[Pulpcore::Config]: Scheduling refresh of Class[Pulpcore::Database]
  Info: Class[Pulpcore::Config]: Scheduling refresh of Class[Pulpcore::Service]
  Info: Class[Pulpcore::Database]: Scheduling refresh of Postgresql::Server::Db[pulpcore]
  Info: Class[Pulpcore::Database]: Scheduling refresh of Pulpcore::Admin[migrate --noinput]
  Info: Postgresql::Server::Db[pulpcore]: Scheduling refresh of Postgresql::Server::Database[pulpcore]
  Info: Postgresql::Server::Db[pulpcore]: Scheduling refresh of Postgresql::Server::Role[pulp]
  Info: Postgresql::Server::Db[pulpcore]: Scheduling refresh of Postgresql::Server::Database_grant[GRANT pulp - ALL - pulpcore]
  Info: Postgresql::Server::Role[pulp]: Scheduling refresh of Postgresql_psql[CREATE ROLE pulp ENCRYPTED PASSWORD ****]
  Info: Postgresql::Server::Role[pulp]: Scheduling refresh of Postgresql_psql[ALTER ROLE "pulp" NOSUPERUSER]
  Info: Postgresql::Server::Role[pulp]: Scheduling refresh of Postgresql_psql[ALTER ROLE "pulp" NOCREATEDB]
  Info: Postgresql::Server::Role[pulp]: Scheduling refresh of Postgresql_psql[ALTER ROLE "pulp" NOCREATEROLE]
  Info: Postgresql::Server::Role[pulp]: Scheduling refresh of Postgresql_psql[ALTER ROLE "pulp" LOGIN]
  Info: Postgresql::Server::Role[pulp]: Scheduling refresh of Postgresql_psql[ALTER ROLE "pulp" INHERIT]
  Info: Postgresql::Server::Role[pulp]: Scheduling refresh of Postgresql_psql[ALTER ROLE "pulp" NOREPLICATION]
  Info: Postgresql::Server::Role[pulp]: Scheduling refresh of Postgresql_psql[ALTER ROLE "pulp" CONNECTION LIMIT -1]
  Info: Postgresql::Server::Role[pulp]: Scheduling refresh of Postgresql_psql[ALTER ROLE pulp ENCRYPTED PASSWORD ****]
  Info: Postgresql::Server::Database_grant[GRANT pulp - ALL - pulpcore]: Scheduling refresh of Postgresql::Server::Grant[database:GRANT pulp - ALL - pulpcore]
  Notice: /Stage[main]/Pulpcore::Database/Postgresql::Server::Db[pulpcore]/Postgresql::Server::Role[pulp]/Postgresql_psql[CREATE ROLE pulp ENCRYPTED PASSWORD ****]: Triggered 'refresh' from 1 event
  Notice: /Stage[main]/Pulpcore::Database/Postgresql::Server::Db[pulpcore]/Postgresql::Server::Role[pulp]/Postgresql_psql[ALTER ROLE "pulp" NOSUPERUSER]: Triggered 'refresh' from 1 event
  Notice: /Stage[main]/Pulpcore::Database/Postgresql::Server::Db[pulpcore]/Postgresql::Server::Role[pulp]/Postgresql_psql[ALTER ROLE "pulp" NOCREATEDB]: Triggered 'refresh' from 1 event
  Notice: /Stage[main]/Pulpcore::Database/Postgresql::Server::Db[pulpcore]/Postgresql::Server::Role[pulp]/Postgresql_psql[ALTER ROLE "pulp" NOCREATEROLE]: Triggered 'refresh' from 1 event
  Notice: /Stage[main]/Pulpcore::Database/Postgresql::Server::Db[pulpcore]/Postgresql::Server::Role[pulp]/Postgresql_psql[ALTER ROLE "pulp" LOGIN]: Triggered 'refresh' from 1 event
  Notice: /Stage[main]/Pulpcore::Database/Postgresql::Server::Db[pulpcore]/Postgresql::Server::Role[pulp]/Postgresql_psql[ALTER ROLE "pulp" INHERIT]: Triggered 'refresh' from 1 event
  Notice: /Stage[main]/Pulpcore::Database/Postgresql::Server::Db[pulpcore]/Postgresql::Server::Role[pulp]/Postgresql_psql[ALTER ROLE "pulp" NOREPLICATION]: Triggered 'refresh' from 1 event
  Notice: /Stage[main]/Pulpcore::Database/Postgresql::Server::Db[pulpcore]/Postgresql::Server::Role[pulp]/Postgresql_psql[ALTER ROLE "pulp" CONNECTION LIMIT -1]: Triggered 'refresh' from 1 event
  Notice: /Stage[main]/Pulpcore::Database/Postgresql::Server::Db[pulpcore]/Postgresql::Server::Role[pulp]/Postgresql_psql[ALTER ROLE pulp ENCRYPTED PASSWORD ****]: Triggered 'refresh' from 1 event
  Info: Postgresql::Server::Database[pulpcore]: Scheduling refresh of Postgresql_psql[CREATE DATABASE "pulpcore"]
  Info: Postgresql::Server::Database[pulpcore]: Scheduling refresh of Postgresql_psql[REVOKE CONNECT ON DATABASE "pulpcore" FROM public]
  Info: Postgresql::Server::Database[pulpcore]: Scheduling refresh of Postgresql_psql[UPDATE pg_database SET datistemplate = false WHERE datname = 'pulpcore']
  Notice: /Stage[main]/Pulpcore::Database/Postgresql::Server::Db[pulpcore]/Postgresql::Server::Database[pulpcore]/Postgresql_psql[CREATE DATABASE "pulpcore"]: Triggered 'refresh' from 1 event
  Info: /Stage[main]/Pulpcore::Database/Postgresql::Server::Db[pulpcore]/Postgresql::Server::Database[pulpcore]/Postgresql_psql[CREATE DATABASE "pulpcore"]: Scheduling refresh of Postgresql_psql[REVOKE CONNECT ON DATABASE "pulpcore" FROM public]
  Notice: /Stage[main]/Pulpcore::Database/Postgresql::Server::Db[pulpcore]/Postgresql::Server::Database[pulpcore]/Postgresql_psql[REVOKE CONNECT ON DATABASE "pulpcore" FROM public]: Triggered 'refresh' from 2 events
  Notice: /Stage[main]/Pulpcore::Database/Postgresql::Server::Db[pulpcore]/Postgresql::Server::Database[pulpcore]/Postgresql_psql[UPDATE pg_database SET datistemplate = false WHERE datname = 'pulpcore']: Triggered 'refresh' from 1 event
  Info: Postgresql::Server::Grant[database:GRANT pulp - ALL - pulpcore]: Scheduling refresh of Postgresql_psql[grant:database:GRANT pulp - ALL - pulpcore]
  Notice: /Stage[main]/Pulpcore::Database/Postgresql::Server::Db[pulpcore]/Postgresql::Server::Database_grant[GRANT pulp - ALL - pulpcore]/Postgresql::Server::Grant[database:GRANT pulp - ALL - pulpcore]/Postgresql_psql[grant:database:GRANT pulp - ALL - pulpcore]: Triggered 'refresh' from 1 event
  Info: Pulpcore::Admin[migrate --noinput]: Scheduling refresh of Exec[python3-django-admin migrate --noinput]
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]: Triggered 'refresh' from 1 event
  Info: Class[Pulpcore::Service]: Scheduling refresh of Systemd::Unit_file[pulpcore-api.service]
  Info: Class[Pulpcore::Service]: Scheduling refresh of Systemd::Unit_file[pulpcore-content.service]
  Info: Class[Pulpcore::Service]: Scheduling refresh of Systemd::Unit_file[pulpcore-resource-manager.service]
  Info: Class[Pulpcore::Service]: Scheduling refresh of Systemd::Unit_file[pulpcore-worker@.service]
  Info: Class[Pulpcore::Service]: Scheduling refresh of Service[pulpcore-worker@1]
  Info: Class[Pulpcore::Service]: Scheduling refresh of Service[pulpcore-worker@2]
  Info: Systemd::Unit_file[pulpcore-api.service]: Scheduling refresh of Service[pulpcore-api.service]
  Info: Systemd::Unit_file[pulpcore-content.service]: Scheduling refresh of Service[pulpcore-content.service]
  Info: Systemd::Unit_file[pulpcore-resource-manager.service]: Scheduling refresh of Service[pulpcore-resource-manager.service]
  Notice: /Stage[main]/Pulpcore::Service/Service[pulpcore-worker@1]: Triggered 'refresh' from 1 event
  Notice: /Stage[main]/Pulpcore::Service/Service[pulpcore-worker@2]: Triggered 'refresh' from 1 event
  Notice: /Stage[main]/Pulpcore::Service/Systemd::Unit_file[pulpcore-api.service]/Service[pulpcore-api.service]: Triggered 'refresh' from 1 event
  Notice: /Stage[main]/Pulpcore::Service/Systemd::Unit_file[pulpcore-content.service]/Service[pulpcore-content.service]: Triggered 'refresh' from 1 event
  Notice: /Stage[main]/Pulpcore::Service/Systemd::Unit_file[pulpcore-resource-manager.service]/Service[pulpcore-resource-manager.service]: Triggered 'refresh' from 1 event
  Notice: Applied catalog in 16.52 seconds

centos8-64-1 executed in 27.49 seconds
Exited: 2
    applies a second time without changes (FAILED - 1)
  Service "httpd"

wbclark · 2020-03-13T17:02:16Z

quick question for @ekohl -- do we need to support Puppet 5 with el8 or is it safe to assume el8 will be at least Puppet 6?

ekohl · 2020-03-13T17:03:40Z

We should support Puppet 5 as well and there's no plan to drop that. Satellite also ships with Puppet 5 and there are some minor differences so I'd still treat it as first class supported.

ekohl

Please add it to https://github.com/theforeman/puppet-pulpcore/blob/master/.sync.yml so it won't be overwritten in the next modulesync.

ekohl · 2020-03-13T17:22:58Z

It was fine to change .travis.yml, but adding it to .sync.yml was also needed.

wbclark · 2020-03-13T17:55:57Z

  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns:     import gridfs

  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns: ModuleNotFoundError: No module named 'gridfs'

  Error: 'python3-django-admin migrate --noinput' returned 1 instead of one of [0]

  Error: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns: change from 'notrun' to ['0'] failed: 'python3-django-admin migrate --noinput' returned 1 instead of one of [0]

ekohl · 2020-03-13T18:10:10Z

The acceptance tests are doing their job :) Looks like a packaging issue.

ehelms · 2020-03-16T23:31:34Z

When I run the tests locally as EL7 or EL8 I am seeing an selinux relabeling happening that breaks idempotency, e.g.

  Notice: /Stage[main]/Redis::Config/File[/var/opt/rh/rh-redis5/lib/redis]/seltype: seltype changed 'var_t' to 'redis_var_lib_t'
  Notice: /Stage[main]/Pulpcore::Config/File[/var/lib/pulp/tmp]/seltype: seltype changed 'var_lib_t' to 'pulpcore_var_lib_t'

ehelms · 2020-03-17T17:36:26Z

I dug into the EL8 gridfs issue. The problem lies in what version of pymongo is being installed. In this case, it is installing the version from the python36 because modules will exclude non-modular repositories if those repositories contain updates to a package from a module. To get around this, we need to enable module_hotfixes in our repository definitions. Effectively modules treat repositories that are non-modular as look asides, as a way for you to provide updated versions of packages outside the module itself.

https://dnf.readthedocs.io/en/latest/modularity.html#hotfix-repositories

This diff worked for me (the SELinux idempotency is still an issue):

diff --git a/spec/spec_helper_acceptance.rb b/spec/spec_helper_acceptance.rb
index 5f3338a..afc4e8e 100644
--- a/spec/spec_helper_acceptance.rb
+++ b/spec/spec_helper_acceptance.rb
@@ -42,6 +42,7 @@ RSpec.configure do |c|
                   end
 
         on host, puppet_resource('yumrepo', 'pulpcore', "baseurl=#{baseurl}", 'gpgcheck=0')
+        on host, 'echo "module_hotfixes=1" >> /etc/yum.repos.d/pulpcore.repo'
       end
     end
   end

And if you are wondering, the yumrepo type is not updated to have all the newer dnf configuration options available. So it was either this simple workaround, or using a file resource type.

ekohl · 2020-03-17T18:06:29Z

When I run the tests locally as EL7 or EL8 I am seeing an selinux relabeling happening that breaks idempotency, e.g.

This is indeed a limitation we run into with the installer as well. I've asked around and the issue we thought was tracking it, isn't actually that. I need to file a proper bugreport.

ehelms · 2020-03-17T18:15:00Z

Is there a work around for the bug in our testing we can implement? Installing the packages before the puppet run?

ekohl · 2020-03-17T20:02:35Z

It's what we do for puppet-foreman:
https://github.com/theforeman/puppet-foreman/blob/15c85f46796ab7775a4555549be16bf74e7ddd9e/spec/spec_helper_acceptance.rb#L33-L34

In this case it's too bad that it includes the pulpcore-selinux package, which is normally managed by the package itself.

We don't see it in CI because that's running on Docker which doesn't support SELinux. I guess we could only install the packages in the spec helper if BEAKER_HYPERVISOR != 'docker' to test full coverage.

wbclark · 2020-03-17T20:27:47Z

Is there a work around for the bug in our testing we can implement? Installing the packages before the puppet run?

I have a working fix for the relabeling of /var/lib/pulp/tmp and i'm testing a few approaches to fixing the similar relabeling of some redis scl path.

wbclark · 2020-03-17T21:56:10Z

I opened #78 to ensure relabeling of the cache dir is performed after it is created and therefore will not break idempotence.

The other idempotence breaking relabeling issue is related to the redis scl (and therefore does not affect el8) and I will resolve it by pre-installing the RPM in the el7 test environment per @ekohl 's suggestion.

ekohl · 2020-04-02T19:13:23Z

I've changed the PR title to better cover the intent.

wbclark · 2020-05-06T15:06:14Z

This is failing on installing the migration plugin on EL8.

AFAIK there will be no pulp2 on EL8 ever, so I should change the tests to not even attempt installing the plugin on EL8.

ekohl · 2020-05-06T16:20:44Z

AFAIK there will be no pulp2 on EL8 ever, so I should change the tests to not even attempt installing the plugin on EL8.

AFAIK you do install the plugin to import content from Pulp 2 servers.

wbclark · 2020-05-06T16:44:50Z

AFAIK there will be no pulp2 on EL8 ever, so I should change the tests to not even attempt installing the plugin on EL8.

AFAIK you do install the plugin to import content from Pulp 2 servers.

Thanks. Latest changes should resolve the issue that was blocking it. I've rebased and I'm testing it out.

wbclark · 2020-05-07T02:20:24Z

Migration plugin should be installable on el8 now

wbclark · 2020-05-07T03:09:24Z

Tests are 🍏

ekohl · 2020-05-07T09:31:06Z

spec/setup_acceptance_node.pp

-  gpgcheck => 0,
-  include  => 'file:///etc/yum/pulpcore.conf',
+file { '/etc/yum.repos.d/pulpcore.repo':
+  ensure  => 'present',


In this case it doesn't matter that much, but I've learned to be explicit with the file type and state whether it's a file or directory.

Suggested change

ensure => 'present',

ensure => 'file',

ekohl · 2020-05-07T12:51:16Z

spec/setup_acceptance_node.pp

 }

 # Needed as a workaround for idempotency
 if $facts['os']['selinux']['enabled'] {
  package { 'pulpcore-selinux':
-    ensure  => installed,
-    require => Yumrepo['pulpcore'],
+    ensure  => file,


I think you meant to change the file resource, not the package one.

.....I shouldn't push anything before I've had my coffee

ekohl requested changes Mar 11, 2020

View reviewed changes

wbclark force-pushed the el8_spec branch 2 times, most recently from bc50204 to eb08b03 Compare March 13, 2020 16:58

ekohl requested changes Mar 13, 2020

View reviewed changes

wbclark force-pushed the el8_spec branch from eb08b03 to 5d3557a Compare March 13, 2020 17:10

wbclark force-pushed the el8_spec branch from 5d3557a to 590e9b4 Compare March 13, 2020 17:24

wbclark force-pushed the el8_spec branch 3 times, most recently from 0a9892e to e0ee399 Compare March 17, 2020 21:29

wbclark force-pushed the el8_spec branch from e0ee399 to 8244fb7 Compare April 2, 2020 17:39

ekohl changed the title ~~Refs #29190 - Allow spec tests to run on el8~~ Refs #29190 - Support EL8 Apr 2, 2020

zjhuntin added the Waiting on contributor label Apr 9, 2020

wbclark mentioned this pull request Apr 16, 2020

Modulesync: use voxpupuli helper gems #90

Merged

wbclark force-pushed the el8_spec branch from 8244fb7 to 51c8e18 Compare May 6, 2020 15:12

theforeman-bot added Needs re-review Needs testing and removed Waiting on contributor labels May 6, 2020

wbclark force-pushed the el8_spec branch from 51c8e18 to af4f2e1 Compare May 6, 2020 16:41

wbclark force-pushed the el8_spec branch from af4f2e1 to 4cd19fa Compare May 7, 2020 02:17

wbclark requested a review from ekohl May 7, 2020 03:10

wbclark changed the title ~~Refs #29190 - Support EL8~~ Fixes #29190 - Support EL8 May 7, 2020

ekohl approved these changes May 7, 2020

View reviewed changes

theforeman-bot removed the Needs re-review label May 7, 2020

wbclark force-pushed the el8_spec branch from 4cd19fa to 9c88b63 Compare May 7, 2020 12:45

ekohl reviewed May 7, 2020

View reviewed changes

Fixes #29190 - support el8

eaab1f9

wbclark force-pushed the el8_spec branch from 9c88b63 to eaab1f9 Compare May 7, 2020 12:59

ehelms merged commit 87897cf into theforeman:master May 7, 2020

ekohl added the Enhancement New feature or request label May 13, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fixes #29190 - Support EL8 #77

Fixes #29190 - Support EL8 #77

wbclark commented Mar 10, 2020

ekohl left a comment

wbclark commented Mar 12, 2020

ekohl commented Mar 12, 2020

ehelms commented Mar 13, 2020

wbclark commented Mar 13, 2020

wbclark commented Mar 13, 2020

ekohl commented Mar 13, 2020

ekohl left a comment

ekohl commented Mar 13, 2020

wbclark commented Mar 13, 2020

ekohl commented Mar 13, 2020

ehelms commented Mar 16, 2020

ehelms commented Mar 17, 2020

ekohl commented Mar 17, 2020

ehelms commented Mar 17, 2020

ekohl commented Mar 17, 2020

wbclark commented Mar 17, 2020 •

edited

Loading

wbclark commented Mar 17, 2020 •

edited

Loading

ekohl commented Apr 2, 2020

wbclark commented May 6, 2020

ekohl commented May 6, 2020

wbclark commented May 6, 2020

wbclark commented May 7, 2020

wbclark commented May 7, 2020

ekohl May 7, 2020

ekohl May 7, 2020

wbclark May 7, 2020

Fixes #29190 - Support EL8 #77

Fixes #29190 - Support EL8 #77

Conversation

wbclark commented Mar 10, 2020

ekohl left a comment

Choose a reason for hiding this comment

wbclark commented Mar 12, 2020

ekohl commented Mar 12, 2020

ehelms commented Mar 13, 2020

wbclark commented Mar 13, 2020

wbclark commented Mar 13, 2020

ekohl commented Mar 13, 2020

ekohl left a comment

Choose a reason for hiding this comment

ekohl commented Mar 13, 2020

wbclark commented Mar 13, 2020

ekohl commented Mar 13, 2020

ehelms commented Mar 16, 2020

ehelms commented Mar 17, 2020

ekohl commented Mar 17, 2020

ehelms commented Mar 17, 2020

ekohl commented Mar 17, 2020

wbclark commented Mar 17, 2020 • edited Loading

wbclark commented Mar 17, 2020 • edited Loading

ekohl commented Apr 2, 2020

wbclark commented May 6, 2020

ekohl commented May 6, 2020

wbclark commented May 6, 2020

wbclark commented May 7, 2020

wbclark commented May 7, 2020

ekohl May 7, 2020

Choose a reason for hiding this comment

ekohl May 7, 2020

Choose a reason for hiding this comment

wbclark May 7, 2020

Choose a reason for hiding this comment

wbclark commented Mar 17, 2020 •

edited

Loading

wbclark commented Mar 17, 2020 •

edited

Loading