Remove Pulp 2 functionality #271

ekohl · 2020-07-02T12:36:46Z

Includes #268

Various TODOs in the code that should be looked at.

ehelms · 2020-07-03T12:53:21Z

manifests/pub_dir.pp

-    additional_includes => ["${apache::confd_dir}/pulp-vhosts80/*.conf"],
-    custom_fragment     => template('foreman_proxy_content/httpd_pub.erb'),
-  }
-}


With Pulp 2 removed, we still need the pub dir (but not necessarily this implementation of it). This is responsible for some non-Pulp related things:

serving katello-consumer-rpm

serving CA cert

serving katello-client-bootstrap

Ohh, I see you moved that up into init

ehelms · 2020-07-06T18:30:47Z

I am seeing the following testing this with the pulpcore change:

Evaluation Error: Error while evaluating a Resource Statement, Evaluation Error: Error while evaluating a Resource Statement, Duplicate declaration: Selboolean[httpd_can_network_connect] is already declared at (file: /usr/share/foreman-installer/modules/foreman/manifests/config/apache.pp, line: 226); cannot redeclare (file: /usr/share/foreman-installer/modules/selinux/manifests/boolean.pp, line: 41) (file: /usr/share/foreman-installer/modules/selinux/manifests/boolean.pp, line: 41, column: 5) (file: /usr/share/foreman-installer/modules/pulpcore/manifests/apache.pp, line: 120) on node pipeline-katello-server-nightly-centos8.woodford.example.com

ehelms · 2020-07-06T18:52:12Z

There is also a failure in creating the vhost fragments for Pulpcore, looks like the vhost priority needs to be passed in if attaching to another vhost.

[ WARN 2020-07-06T18:31:47 main]  /Stage[main]/Pulpcore::Apache/Pulpcore::Apache::Fragment[pulpcore]/Apache::Vhost::Fragment[pulpcore-http-pulpcore]/Concat::Fragment[foreman-pulpcore-http-pulpcore]/Concat_fragment[foreman-pulpcore-http-pulpcore]: Target Concat_file with path or title '10-foreman.conf' or tag '10-foreman.conf not found in the catalog
[ WARN 2020-07-06T18:31:47 main]  /Stage[main]/Pulpcore::Apache/Pulpcore::Apache::Fragment[pulpcore]/Apache::Vhost::Fragment[pulpcore-https-pulpcore]/Concat::Fragment[foreman-ssl-pulpcore-https-pulpcore]/Concat_fragment[foreman-ssl-pulpcore-https-pulpcore]: Target Concat_file with path or title '10-foreman-ssl.conf' or tag '10-foreman-ssl.conf not found in the catalog
[ WARN 2020-07-06T18:31:47 main]  /Stage[main]/Foreman_proxy_content/Pulpcore::Apache::Fragment[pub_dir]/Apache::Vhost::Fragment[pulpcore-http-pub_dir]/Concat::Fragment[foreman-pulpcore-http-pub_dir]/Concat_fragment[foreman-pulpcore-http-pub_dir]: Target Concat_file with path or title '10-foreman.conf' or tag '10-foreman.conf not found in the catalog
[ WARN 2020-07-06T18:31:47 main]  /Stage[main]/Foreman_proxy_content/Pulpcore::Apache::Fragment[pub_dir]/Apache::Vhost::Fragment[pulpcore-https-pub_dir]/Concat::Fragment[foreman-ssl-pulpcore-https-pub_dir]/Concat_fragment[foreman-ssl-pulpcore-https-pub_dir]: Target Concat_file with path or title '10-foreman-ssl.conf' or tag '10-foreman-ssl.conf not found in the catalog
[ WARN 2020-07-06T18:31:47 main]  /Stage[main]/Foreman_proxy_content/Pulpcore::Apache::Fragment[gpg_key_proxy]/Apache::Vhost::Fragment[pulpcore-https-gpg_key_proxy]/Concat::Fragment[foreman-ssl-pulpcore-https-gpg_key_proxy]/Concat_fragment[foreman-ssl-pulpcore-https-gpg_key_proxy]: Target Concat_file with path or title '10-foreman-ssl.conf' or tag '10-foreman-ssl.conf not found in the catalog
[ WARN 2020-07-06T18:31:47 main]  /Stage[main]/Pulpcore::Plugin::Container/Pulpcore::Plugin[container]/Pulpcore::Apache::Fragment[plugin-container]/Apache::Vhost::Fragment[pulpcore-https-plugin-container]/Concat::Fragment[foreman-ssl-pulpcore-https-plugin-container]/Concat_fragment[foreman-ssl-pulpcore-https-plugin-container]: Target Concat_file with path or title '10-foreman-ssl.conf' or tag '10-foreman-ssl.conf not found in the catalog
[ WARN 2020-07-06T18:31:47 main]  /Stage[main]/Pulpcore::Plugin::File/Pulpcore::Plugin[file]/Pulpcore::Apache::Fragment[plugin-file]/Apache::Vhost::Fragment[pulpcore-http-plugin-file]/Concat::Fragment[foreman-pulpcore-http-plugin-file]/Concat_fragment[foreman-pulpcore-http-plugin-file]: Target Concat_file with path or title '10-foreman.conf' or tag '10-foreman.conf not found in the catalog
[ WARN 2020-07-06T18:31:47 main]  /Stage[main]/Pulpcore::Plugin::File/Pulpcore::Plugin[file]/Pulpcore::Apache::Fragment[plugin-file]/Apache::Vhost::Fragment[pulpcore-https-plugin-file]/Concat::Fragment[foreman-ssl-pulpcore-https-plugin-file]/Concat_fragment[foreman-ssl-pulpcore-https-plugin-file]: Target Concat_file with path or title '10-foreman-ssl.conf' or tag '10-foreman-ssl.conf not found in the catalog
[ WARN 2020-07-06T18:31:47 main]  /Stage[main]/Pulpcore::Plugin::Rpm/Pulpcore::Plugin[rpm]/Pulpcore::Apache::Fragment[plugin-rpm]/Apache::Vhost::Fragment[pulpcore-http-plugin-rpm]/Concat::Fragment[foreman-pulpcore-http-plugin-rpm]/Concat_fragment[foreman-pulpcore-http-plugin-rpm]: Target Concat_file with path or title '10-foreman.conf' or tag '10-foreman.conf not found in the catalog
[ WARN 2020-07-06T18:31:47 main]  /Stage[main]/Pulpcore::Plugin::Rpm/Pulpcore::Plugin[rpm]/Pulpcore::Apache::Fragment[plugin-rpm]/Apache::Vhost::Fragment[pulpcore-https-plugin-rpm]/Concat::Fragment[foreman-ssl-pulpcore-https-plugin-rpm]/Concat_fragment[foreman-ssl-pulpcore-https-plugin-rpm]: Target Concat_file with path or title '10-foreman-ssl.conf' or tag '10-foreman-ssl.conf not found in the catalog

ekohl · 2020-07-06T19:03:15Z

I have not even tried to run it myself. It was only me quickly going over the code and see what would roughly be the steps and things that should remain.

I am seeing the following testing this with the pulpcore change:

Evaluation Error: Error while evaluating a Resource Statement, Evaluation Error: Error while evaluating a Resource Statement, Duplicate declaration: Selboolean[httpd_can_network_connect] is already declared at (file: /usr/share/foreman-installer/modules/foreman/manifests/config/apache.pp, line: 226); cannot redeclare (file: /usr/share/foreman-installer/modules/selinux/manifests/boolean.pp, line: 41) (file: /usr/share/foreman-installer/modules/selinux/manifests/boolean.pp, line: 41, column: 5) (file: /usr/share/foreman-installer/modules/pulpcore/manifests/apache.pp, line: 120) on node pipeline-katello-server-nightly-centos8.woodford.example.com

I suspected as much, see the TODO there. Perhaps it's better to make it part of the SELinux policy. Same for Foreman itself. theforeman/puppet-foreman#849 is another example of someone who ran into this.

There is also a failure in creating the vhost fragments for Pulpcore, looks like the vhost priority needs to be passed in if attaching to another vhost.

Looks like Foreman uses the 05 priority. Perhaps we do need to expose it as a top level parameter. Now you can set it via Hiera.

ehelms · 2020-07-06T19:14:10Z

I don't expect this to fully work yet :) But there is enough work that needs this that I am helpin speed up the testing.

Looks like Foreman uses the 05 priority. Perhaps we do need to expose it as a top level parameter. Now you can set it via Hiera.

From testing this, it's available to be retrieved. We should need to pass it in here -- https://github.com/theforeman/puppet-foreman_proxy_content/pull/271/files#diff-60ae41fd0a31977447947f59940ee9a4R125 as something like apache_vhost_priority => $foreman::config::apache::priority, to a parameter on puppet-pulpcore. That worked for me locally.

ehelms · 2020-07-06T19:22:44Z

manifests/init.pp

+  Stdlib::Port $reverse_proxy_port = 8443,
+
+  Optional[String] $rhsm_hostname = undef,
+  String $rhsm_url = '/rhsm', # TODO: Not really a URL


Probably shouldn't be configurable to be honest, the API is the API.

Having it live in a variable is generally a good thing and I think this still has some room to split functionality in smaller classes. Then you can make it a parameter on the private class API but not expose it in the public class API.

I'm good with it being a variable, I meant the perceived intent that a user can and should change the value is wrong. This is a hard-coded API in the Katello plugin so having this be changeable by someone using the public interface of this module can result in incorrect behavior.

Agreed, but I'm first going to focus on the Pulp parts. It should probably be done as part of #272 since that does introduce the private API I was thinking about.

manifests/init.pp

ekohl · 2020-07-07T10:11:43Z

Updated:

Initial version which allows deploying without Foreman via $attach_to_foreman_vhost
Pass the vhost priority
Remove $parent_fqdn

ehelms · 2020-07-09T20:37:42Z

Could you update this next time around to point to https://projects.theforeman.org/issues/30359

This is handled in the spec helper and only leads to a duplicate report.

When Pulp 2 parameters are set, Pulp 3 parameters should also be set. This disabled Pulp 3 in many cases leading to faster spec tests.

With RSpec contexts can be nested and they'll inherit the parent let blocks. The descriptions will be included as well.

ekohl · 2020-12-02T14:55:41Z

Rebased just to see how it would look. This still assumes Pulp 2 and qpid are removed at the same time.

ekohl · 2020-12-14T14:08:52Z

Closing in favor of #306

theforeman-bot added Needs testing Not yet reviewed labels Jul 2, 2020

ehelms reviewed Jul 3, 2020

View reviewed changes

ehelms reviewed Jul 6, 2020

View reviewed changes

manifests/init.pp Outdated Show resolved Hide resolved

ekohl mentioned this pull request Jul 7, 2020

Add full Apache management theforeman/puppet-pulpcore#105

Merged

ekohl force-pushed the remove-pulp2 branch from 7846684 to 77c6042 Compare July 7, 2020 10:09

ehelms mentioned this pull request Jul 9, 2020

Refs #28961: Disable Crane when Pulp 3 is serving container content #239

Closed

ehelms mentioned this pull request Jul 9, 2020

Fixes #30360: Drop Pulp 2 and Qpid theforeman/puppet-katello#356

Closed

ekohl force-pushed the remove-pulp2 branch from 77c6042 to 70710fa Compare October 18, 2020 13:26

ekohl added 4 commits December 2, 2020 15:40

Remove coverage spec

a9f45dd

This is handled in the spec helper and only leads to a duplicate report.

Always set pulpcore_enabled in specs

00b822d

When Pulp 2 parameters are set, Pulp 3 parameters should also be set. This disabled Pulp 3 in many cases leading to faster spec tests.

Use RSpec's context nesting

00992c7

With RSpec contexts can be nested and they'll inherit the parent let blocks. The descriptions will be included as well.

Remove Pulp 2 functionality

ca9661c

ekohl force-pushed the remove-pulp2 branch from 70710fa to ca9661c Compare December 2, 2020 14:52

ehelms mentioned this pull request Dec 10, 2020

Refs #31614: Drop Pulp 2, Pulpcore only #306

Merged

ekohl closed this Dec 14, 2020

ekohl deleted the remove-pulp2 branch December 14, 2020 14:08

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove Pulp 2 functionality #271

Remove Pulp 2 functionality #271

ekohl commented Jul 2, 2020

ehelms Jul 3, 2020

ehelms Jul 3, 2020

ehelms commented Jul 6, 2020

ehelms commented Jul 6, 2020 •

edited

ekohl commented Jul 6, 2020

ehelms commented Jul 6, 2020

ehelms Jul 6, 2020

ekohl Jul 7, 2020

ehelms Jul 7, 2020

ekohl Jul 7, 2020

ekohl commented Jul 7, 2020

ehelms commented Jul 9, 2020

ekohl commented Dec 2, 2020

ekohl commented Dec 14, 2020

Remove Pulp 2 functionality #271

Remove Pulp 2 functionality #271

Conversation

ekohl commented Jul 2, 2020

ehelms Jul 3, 2020

Choose a reason for hiding this comment

ehelms Jul 3, 2020

Choose a reason for hiding this comment

ehelms commented Jul 6, 2020

ehelms commented Jul 6, 2020 • edited

ekohl commented Jul 6, 2020

ehelms commented Jul 6, 2020

ehelms Jul 6, 2020

Choose a reason for hiding this comment

ekohl Jul 7, 2020

Choose a reason for hiding this comment

ehelms Jul 7, 2020

Choose a reason for hiding this comment

ekohl Jul 7, 2020

Choose a reason for hiding this comment

ekohl commented Jul 7, 2020

ehelms commented Jul 9, 2020

ekohl commented Dec 2, 2020

ekohl commented Dec 14, 2020

ehelms commented Jul 6, 2020 •

edited