Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Drop Pulpcore 3.14 & 3.15, move to 3.16 and 3.17 #249

Merged
merged 0 commits into from
Mar 21, 2022

Conversation

ekohl
Copy link
Member

@ekohl ekohl commented Mar 9, 2022

No description provided.

@ekohl
Copy link
Member Author

ekohl commented Mar 9, 2022

I just realized 3.17 states (https://docs.pulpproject.org/pulpcore/changes.html):

The pulpcore-worker binary no longer accepts the --resource-manager flag. There is no resource manager anymore, so this flag is no longer needed. #9327

This still needs to be addressed in this PR. In particular with 3.16 implications.

@@ -65,8 +65,7 @@
end

describe curl_command("https://#{host_inventory['fqdn']}/pulp/api/v3/", cacert: "#{certdir}/ca-cert.pem") do
its(:response_code) { is_expected.to eq(200) }
its(:body) { is_expected.not_to contain('artifacts_list') }
its(:response_code) { is_expected.to eq(403) }
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do wonder if this is correct. Also, why is the HTTP 200 code in the other tests (plugin, content route spec).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't look correct to me :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On the other hand, what it this supposed to be? The api root, listing all the endpoints? Does it work if you pass a cert/key?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So what I do find odd is that it does pass in the other tests. I haven't looked into the details yet, but the inconsistency strikes me as odd.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's a side-effect of pulp/pulpcore#1593 (and I've asked in pulp-dev to make sure).

If you want to test an endpoint w/o auth, use /pulp/api/v3/status/ as we do in line 57.

That said, I am not really sure what this test is supposed to test. It was added as part of a60df53 where we implemented HTTPS handling inside this module, but it isn't saying what it's testing :(

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FTR, I've opened pulp/pulpcore#2340 as Grant said on IRC this was probably not an intentional change.
I do however stand by my analysis that this test as such is not giving any value to this module (while the /status one does) and can be dropped.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea was mostly that it showed a difference between authenticated and non-authenticated. With that you can see that certificate authentication works. That's something this module sets up. If you have a better idea of how to test auth, I'm open to suggestions.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But that's exactly what it did not test, as access to / was explicitly allowed without auth (just the content differed).

I would suggest hitting a real API endpoint like /users or /artifacts or /tasks. Without auth you get status=403 and a JSON reply of {"detail":"Authentication credentials were provided."}, with you get status=200 and some actual content.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But that's exactly what it did not test, as access to / was explicitly allowed without auth (just the content differed).

I disagree it did not test it. By checking the difference in output I believe I was able to test it. You may be right that it happened to be a side effect. I'd be happy to merge a follow up PR.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ekohl
Copy link
Member Author

ekohl commented Mar 10, 2022

The failures look like bugs in Pulp 3.17:

Info: Pulpcore::Admin[migrate --noinput]: Scheduling refresh of Exec[pulpcore-manager migrate --noinput]
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns: /opt/theforeman/tfm-pulpcore/root/usr/lib64/python3.8/site-packages/cryptography/hazmat/bindings/openssl/binding.py:173: CryptographyDeprecationWarning: OpenSSL version 1.0.2 is no longer supported by the OpenSSL project, please upgrade. The next version of cryptography will drop support for it.
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:   warnings.warn(
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns: Traceback (most recent call last):
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:   File "/opt/theforeman/tfm-pulpcore/root/usr/lib/python3.8/site-packages/git/__init__.py", line 87, in <module>
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:     refresh()
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:   File "/opt/theforeman/tfm-pulpcore/root/usr/lib/python3.8/site-packages/git/__init__.py", line 76, in refresh
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:     if not Git.refresh(path=path):
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:   File "/opt/theforeman/tfm-pulpcore/root/usr/lib/python3.8/site-packages/git/cmd.py", line 340, in refresh
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:     raise ImportError(err)
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns: ImportError: Bad git executable.
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns: The git executable must be specified in one of the following ways:
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:     - be included in your $PATH
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:     - be set via $GIT_PYTHON_GIT_EXECUTABLE
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:     - explicitly set via git.refresh()
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns: 
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns: All git commands will error until this is rectified.
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns: 
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns: This initial warning can be silenced or aggravated in the future by setting the
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns: $GIT_PYTHON_REFRESH environment variable. Use one of the following values:
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:     - quiet|q|silence|s|none|n|0: for no warning or exception
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:     - warn|w|warning|1: for a printed warning
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:     - error|e|raise|r|2: for a raised exception
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns: 
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns: Example:
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:     export GIT_PYTHON_REFRESH=quiet
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns: 
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns: 
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns: The above exception was the direct cause of the following exception:
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns: 
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns: Traceback (most recent call last):
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:   File "/opt/theforeman/tfm-pulpcore/root/usr/bin/pulpcore-manager", line 11, in <module>
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:     load_entry_point('pulpcore==3.17.3', 'console_scripts', 'pulpcore-manager')()
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:   File "/opt/theforeman/tfm-pulpcore/root/usr/lib/python3.8/site-packages/pulpcore/app/manage.py", line 11, in manage
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:     execute_from_command_line(sys.argv)
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:   File "/opt/theforeman/tfm-pulpcore/root/usr/lib/python3.8/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:     utility.execute()
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:   File "/opt/theforeman/tfm-pulpcore/root/usr/lib/python3.8/site-packages/django/core/management/__init__.py", line 395, in execute
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:     django.setup()
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:   File "/opt/theforeman/tfm-pulpcore/root/usr/lib/python3.8/site-packages/django/__init__.py", line 24, in setup
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:     apps.populate(settings.INSTALLED_APPS)
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:   File "/opt/theforeman/tfm-pulpcore/root/usr/lib/python3.8/site-packages/django/apps/registry.py", line 122, in populate
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:     app_config.ready()
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:   File "/opt/theforeman/tfm-pulpcore/root/usr/lib/python3.8/site-packages/pulpcore/app/apps.py", line 96, in ready
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:     self.import_viewsets()
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:   File "/opt/theforeman/tfm-pulpcore/root/usr/lib/python3.8/site-packages/pulpcore/app/apps.py", line 140, in import_viewsets
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:     self.viewsets_module = import_module(viewsets_module_name)
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:   File "/opt/rh/rh-python38/root/usr/lib64/python3.8/importlib/__init__.py", line 127, in import_module
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:     return _bootstrap._gcd_import(name[level:], package, level)
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:   File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:   File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:   File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:   File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:   File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:   File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:   File "/opt/theforeman/tfm-pulpcore/root/usr/lib/python3.8/site-packages/pulp_ansible/app/viewsets.py", line 33, in <module>
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:     from pulp_ansible.app.galaxy.mixins import UploadGalaxyCollectionMixin
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:   File "/opt/theforeman/tfm-pulpcore/root/usr/lib/python3.8/site-packages/pulp_ansible/app/galaxy/mixins.py", line 3, in <module>
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:     from pulp_ansible.app.tasks.collections import import_collection
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:   File "/opt/theforeman/tfm-pulpcore/root/usr/lib/python3.8/site-packages/pulp_ansible/app/tasks/collections.py", line 72, in <module>
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:     from git import GitCommandError, Repo
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:   File "/opt/theforeman/tfm-pulpcore/root/usr/lib/python3.8/site-packages/git/__init__.py", line 89, in <module>
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:     raise ImportError('Failed to initialize: {0}'.format(exc)) from exc
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns: ImportError: Failed to initialize: Bad git executable.
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns: The git executable must be specified in one of the following ways:
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:     - be included in your $PATH
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:     - be set via $GIT_PYTHON_GIT_EXECUTABLE
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:     - explicitly set via git.refresh()
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns: 
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns: All git commands will error until this is rectified.
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns: 
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns: This initial warning can be silenced or aggravated in the future by setting the
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns: $GIT_PYTHON_REFRESH environment variable. Use one of the following values:
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:     - quiet|q|silence|s|none|n|0: for no warning or exception
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:     - warn|w|warning|1: for a printed warning
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:     - error|e|raise|r|2: for a raised exception
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns: 
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns: Example:
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:     export GIT_PYTHON_REFRESH=quiet
  Error: 'pulpcore-manager migrate --noinput' returned 1 instead of one of [0]

@ekohl ekohl marked this pull request as ready for review March 10, 2022 11:43
@ekohl
Copy link
Member Author

ekohl commented Mar 10, 2022

Updated the PR to only 3.16 since 3.17 is broken on EL7.

@ehelms ehelms changed the title Drop Pulpcore 3.14 & 3.15, move to 3.16 & 3.17 Drop Pulpcore 3.14 & 3.15, move to 3.16 Mar 10, 2022
manifests/init.pp Outdated Show resolved Hide resolved
@ekohl
Copy link
Member Author

ekohl commented Mar 15, 2022

Note to self: include enabling the pulpcore el8 module in repo.pp.

manifests/repo.pp Outdated Show resolved Hide resolved
@ekohl ekohl force-pushed the update-to-3.16-and-3.17 branch 3 times, most recently from eb6a9cd to 8072d6a Compare March 17, 2022 11:49
spec/classes/repo_spec.rb Outdated Show resolved Hide resolved
@@ -19,6 +20,17 @@
notify => Anchor['pulpcore::repo'],
}

if $facts['os']['release']['major'] == '8' {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if $facts['os']['release']['major'] == '8' {
if $facts['os']['release']['major'] != '7' {

to match tests? (and the idea of future proofing) :)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Odilhao
Copy link
Member

Odilhao commented Mar 17, 2022

Updated the PR to only 3.16 since 3.17 is broken on EL7.

This will probably fix for 3.17 theforeman/pulpcore-packaging#418

@evgeni
Copy link
Member

evgeni commented Mar 18, 2022

Updated the PR to only 3.16 since 3.17 is broken on EL7.

This will probably fix for 3.17 theforeman/pulpcore-packaging#418

It did, thanks!

@ekohl
Copy link
Member Author

ekohl commented Mar 18, 2022

Updated:

README.md Outdated Show resolved Hide resolved
@ehelms ehelms changed the title Drop Pulpcore 3.14 & 3.15, move to 3.16 Drop Pulpcore 3.14 & 3.15, move to 3.16 and 3.17 Mar 21, 2022
@ekohl ekohl merged commit 3dd9f37 into theforeman:master Mar 21, 2022
@ekohl ekohl deleted the update-to-3.16-and-3.17 branch March 21, 2022 15:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants