Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

race condition when deleting a repository and associated distribution #2019

Open
pulpbot opened this issue Jan 17, 2022 · 1 comment
Open

Comments

@pulpbot
Copy link
Member

pulpbot commented Jan 17, 2022

Author: @jlsherrill (jsherril@redhat.com)

Redmine Issue: 8942, https://pulp.plan.io/issues/8942


In our katello tests, we have some 'cleanup' code that:

list and then deletes a repository with a given name
list and then deletes a remote with a given name
list and then deletes a distribution with a given name
wait on all the tasks generated from above

This has worked for quite a while, but after upgrading from pulp-container 2.3 to 2.6, we started seeing an error (ISE) from the server:

LookupError: view not found

when trying to list the distributions on the 3rd step.

I'm not sure if the pulp code is running faster, or slower, but it appears that there is a consistent race condition occuring. I can modify the code to poll the tasks after the repo delete, and it seems to work find (so this isn't blocking us)

Here's the full traceback:


Untitled
From WebUI, 1 Second ago, written in Plain Text.
This paste will self destruct in 1 Day.	
URL https://paste.centos.org/view/99464b52
Embed Show code
Download Paste or View Raw

        Jun 21 16:14:09 devel2 pulpcore-api: pulp [b51142c5bced40989a59403cf6e92fd8]: django.request:ERROR: Internal Server Error: /pulp/api/v3/distributions/container/container/
        Jun 21 16:14:09 devel2 pulpcore-api: Traceback (most recent call last):
        Jun 21 16:14:09 devel2 pulpcore-api: File "/usr/lib/python3.6/site-packages/django/core/handlers/exception.py", line 34, in inner
        Jun 21 16:14:09 devel2 pulpcore-api: response = get_response(request)
        Jun 21 16:14:09 devel2 pulpcore-api: File "/usr/lib/python3.6/site-packages/django/core/handlers/base.py", line 115, in _get_response
        Jun 21 16:14:09 devel2 pulpcore-api: response = self.process_exception_by_middleware(e, request)
        Jun 21 16:14:09 devel2 pulpcore-api: File "/usr/lib/python3.6/site-packages/django/core/handlers/base.py", line 113, in _get_response
        Jun 21 16:14:09 devel2 pulpcore-api: response = wrapped_callback(request, *callback_args, **callback_kwargs)
        Jun 21 16:14:09 devel2 pulpcore-api: File "/usr/lib/python3.6/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
        Jun 21 16:14:09 devel2 pulpcore-api: return view_func(*args, **kwargs)
        Jun 21 16:14:09 devel2 pulpcore-api: File "/usr/lib/python3.6/site-packages/rest_framework/viewsets.py", line 125, in view
        Jun 21 16:14:09 devel2 pulpcore-api: return self.dispatch(request, *args, **kwargs)
        Jun 21 16:14:09 devel2 pulpcore-api: File "/usr/lib/python3.6/site-packages/rest_framework/views.py", line 509, in dispatch
        Jun 21 16:14:09 devel2 pulpcore-api: response = self.handle_exception(exc)
        Jun 21 16:14:09 devel2 pulpcore-api: File "/usr/lib/python3.6/site-packages/rest_framework/views.py", line 469, in handle_exception
        Jun 21 16:14:09 devel2 pulpcore-api: self.raise_uncaught_exception(exc)
        Jun 21 16:14:09 devel2 pulpcore-api: File "/usr/lib/python3.6/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
        Jun 21 16:14:09 devel2 pulpcore-api: raise exc
        Jun 21 16:14:09 devel2 pulpcore-api: File "/usr/lib/python3.6/site-packages/rest_framework/views.py", line 506, in dispatch
        Jun 21 16:14:09 devel2 pulpcore-api: response = handler(request, *args, **kwargs)
        Jun 21 16:14:09 devel2 pulpcore-api: File "/usr/lib/python3.6/site-packages/rest_framework/mixins.py", line 43, in list
        Jun 21 16:14:09 devel2 pulpcore-api: return self.get_paginated_response(serializer.data)
        Jun 21 16:14:09 devel2 pulpcore-api: File "/usr/lib/python3.6/site-packages/rest_framework/serializers.py", line 745, in data
        Jun 21 16:14:09 devel2 pulpcore-api: ret = super().data
        Jun 21 16:14:09 devel2 pulpcore-api: File "/usr/lib/python3.6/site-packages/rest_framework/serializers.py", line 246, in data
        Jun 21 16:14:09 devel2 pulpcore-api: self._data = self.to_representation(self.instance)
        Jun 21 16:14:09 devel2 pulpcore-api: File "/usr/lib/python3.6/site-packages/rest_framework/serializers.py", line 664, in to_representation
        Jun 21 16:14:09 devel2 pulpcore-api: self.child.to_representation(item) for item in iterable
        Jun 21 16:14:09 devel2 pulpcore-api: File "/usr/lib/python3.6/site-packages/rest_framework/serializers.py", line 664, in <listcomp>
        Jun 21 16:14:09 devel2 pulpcore-api: self.child.to_representation(item) for item in iterable
        Jun 21 16:14:09 devel2 pulpcore-api: File "/usr/lib/python3.6/site-packages/rest_framework/serializers.py", line 515, in to_representation
        Jun 21 16:14:09 devel2 pulpcore-api: ret[field.field_name] = field.to_representation(attribute)
        Jun 21 16:14:09 devel2 pulpcore-api: File "/usr/lib/python3.6/site-packages/rest_framework/relations.py", line 401, in to_representation
        Jun 21 16:14:09 devel2 pulpcore-api: url = self.get_url(value, self.view_name, request, format)
        Jun 21 16:14:09 devel2 pulpcore-api: File "/usr/lib/python3.6/site-packages/pulpcore/app/serializers/fields.py", line 210, in get_url
        Jun 21 16:14:09 devel2 pulpcore-api: repo_url = rvr_field.get_url(obj.repository, None, request, *args, **kwargs)
        Jun 21 16:14:09 devel2 pulpcore-api: File "/usr/lib/python3.6/site-packages/pulpcore/app/serializers/fields.py", line 198, in get_url
        Jun 21 16:14:09 devel2 pulpcore-api: return super().get_url(obj, self.view_name, request, *args, **kwargs) + "versions/"
        Jun 21 16:14:09 devel2 pulpcore-api: File "/usr/lib/python3.6/site-packages/pulpcore/app/serializers/base.py", line 236, in get_url
        Jun 21 16:14:09 devel2 pulpcore-api: view_name = self._view_name(obj)
        Jun 21 16:14:09 devel2 pulpcore-api: File "/usr/lib/python3.6/site-packages/pulpcore/app/serializers/base.py", line 231, in _view_name
        Jun 21 16:14:09 devel2 pulpcore-api: return get_view_name_for_model(obj, "detail")
        Jun 21 16:14:09 devel2 pulpcore-api: File "/usr/lib/python3.6/site-packages/pulpcore/app/util.py", line 78, in get_view_name_for_model
        Jun 21 16:14:09 devel2 pulpcore-api: raise LookupError("view not found")
        Jun 21 16:14:09 devel2 pulpcore-api: LookupError: view not found
        Jun 21 16:14:09 devel2 pulpcore-api: pulp [b51142c5bced40989a59403cf6e92fd8]:  - - [21/Jun/2021:16:14:09 +0000] "GET /pulp/api/v3/distributions/container/container/?name=Default_Organization-Test-busybox-library HTTP/1.1" 500 27 "-" "OpenAPI-Generator/2.4.0/ruby"
@stale
Copy link

stale bot commented May 24, 2022

This issue has been marked 'stale' due to lack of recent activity. If there is no further activity, the issue will be closed in another 30 days. Thank you for your contribution!

@stale stale bot added the stale label May 24, 2022
@dralley dralley removed the stale label May 25, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants