Replace enqueue_with_reservation with dispatch for pulpcore 3.14.0 compat #821
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
alikins
changed the title
alikins
force-pushed
the
pulpcore_314_aah_706
branch
from
a4586b3
to
2cbc382
Compare
|
@alikins you need to tell the CI to use the proper versions of pulpcore https://github.com/ansible/galaxy_ng/blob/master/.travis/before_install.sh#L73 and https://github.com/ansible/galaxy_ng/blob/master/.travis/before_install.sh#L103 |
alikins
force-pushed
the
pulpcore_314_aah_706
branch
from
2cbc382
to
351af65
Compare
alikins
added a commit
to alikins/galaxy_ng
that referenced
this pull request
Jul 6, 2021
pulpcore 3.14 replaces the 'rq' based tasking system with 'pulpcore-worker' So no longer need a service for 'resource-manager' on the docker-compose dev env tooling. Removed it from Makefile, the docker-compose.yml for each profile, and the 'start-resource-manager' invoked by the 'resource-manager' docker container. Required PR: ansible#821 No-Issue
alikins
added a commit
to alikins/galaxy_ng
that referenced
this pull request
Jul 6, 2021
See https://github.com/pulp/pulpcore/blob/master/docs/components.rst#distributed-tasking-system-tech-preview Remove use if pulpcore 'resource-manager' for pulp 3.14 pulpcore 3.14 replaces the 'rq' based tasking system with 'pulpcore-worker' So no longer need a service for 'resource-manager' on the docker-compose dev env tooling. Removed it from Makefile, the docker-compose.yml for each profile, and the 'start-resource-manager' invoked by the 'resource-manager' docker container. Required PR: ansible#821 No-Issue
alikins
added a commit
to alikins/galaxy_ng
that referenced
this pull request
Jul 6, 2021
Required PR: ansible#821 No-Issue
alikins
force-pushed
the
pulpcore_314_aah_706
branch
from
351af65
to
7c5626b
Compare
alikins
added a commit
to alikins/galaxy_ng
that referenced
this pull request
Jul 6, 2021
See https://github.com/pulp/pulpcore/blob/master/docs/components.rst#distributed-tasking-system-tech-preview Remove use if pulpcore 'resource-manager' for pulp 3.14 pulpcore 3.14 replaces the 'rq' based tasking system with 'pulpcore-worker' So no longer need a service for 'resource-manager' on the docker-compose dev env tooling. Removed it from Makefile, the docker-compose.yml for each profile, and the 'start-resource-manager' invoked by the 'resource-manager' docker container. Required PR: ansible#821 No-Issue
alikins
added a commit
to alikins/galaxy_ng
that referenced
this pull request
Jul 7, 2021
Required PR: ansible#821 Required PR: ansible#824 Issue: AAH-708
rochacbruno
requested changes
Jul 9, 2021
There was a problem hiding this comment.
The upgrade of dependencies made migration to break
File "/usr/local/lib/python3.6/site-packages/galaxy_ng/app/migrations/0017_populate_repos_and_remotes.py", line 104, in populate_initial_repos
"pulp_type": "ansible.ansible"we need a fix for that migration as part of this PR to make the CI to pass.
__fake__.DoesNotExist: AnsibleDistribution matching query does not exist.
hmm, interesting. Did not expect that. AnsibleDistribution no longer exists? |
|
@alikins the entity exists, but the SQL query made on the migration is failing for some reason. Did we changed something on |
alikins
added a commit
to alikins/galaxy_ng
that referenced
this pull request
Jul 13, 2021
See https://github.com/pulp/pulpcore/blob/master/docs/components.rst#distributed-tasking-system-tech-preview Remove use if pulpcore 'resource-manager' for pulp 3.14 pulpcore 3.14 replaces the 'rq' based tasking system with 'pulpcore-worker' So no longer need a service for 'resource-manager' on the docker-compose dev env tooling. Removed it from Makefile, the docker-compose.yml for each profile, and the 'start-resource-manager' invoked by the 'resource-manager' docker container. Required PR: ansible#821 No-Issue
Primary difference is that enqueue_with_reservation() returns an 'rq' job, and 'rq' support is being removed in pulpcore. The new 'dispatch()' returns a pulpcore.plugin.models.Task object. dispatch() also no longer supports the 'options' arg. pulpcore 3.13 removes 'enqueue_with_reservation' https://docs.pulpproject.org/pulpcore/changes.html#id96 pulpcore 3.12.0 deprecated 'enqueue_with_reservation' https://docs.pulpproject.org/pulpcore/changes.html#id152 See https://docs.pulpproject.org/pulpcore/plugins/api-reference/tasking.html#pulpcore.plugin.tasking.dispatch for docs on dispatch(). More explaination of the diff between enqueue_with_reservation() and dispatch() can be seen at: pulp/pulpcore@753e4dd and https://pulp.plan.io/issues/8496 Issue: AAH-706
See https://github.com/pulp/pulpcore/blob/master/docs/components.rst#distributed-tasking-system-tech-preview Remove use if pulpcore 'resource-manager' for pulp 3.14 pulpcore 3.14 replaces the 'rq' based tasking system with 'pulpcore-worker' So no longer need a service for 'resource-manager' on the docker-compose dev env tooling. Removed it from Makefile, the docker-compose.yml for each profile, and the 'start-resource-manager' invoked by the 'resource-manager' docker container. Required PR: ansible#821 No-Issue
Not sure why, but replacing the Models.objects with one using the 'db_alias', AnsibleDistribution.objects.get() and get_or_create() fail with invalid query which raises a DoesNotExist from a get_or_create() Then it seems like something in the error handling is walking over the models and triggering some recursion storm that seems to get stuck in django lifecycle land. For more detail, see: https://issues.redhat.com/browse/AAH-706?focusedCommentId=16423461&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16423461 Issue: AAH-706
Issue: AAH-706
alikins
force-pushed
the
pulpcore_314_aah_706
branch
from
1b3898a
to
aaf642d
Compare
Issue: AAH-706
Issue: AAH-706
'enqueue_with_reservation' has been replaced with 'dispatch' in pulpcore 3.14. So stop mocking it. Issue: AAH-706
Based on pulp/pulp_ansible@946c356 some of the generated API names have changed. Update to use the (correct?) new name. Issues: AAH-706
rochacbruno
approved these changes
Jul 13, 2021
There was a problem hiding this comment.
LGTM
Waiting to hear from @cutwater related to the appSRE failure.
chouseknecht
pushed a commit
that referenced
this pull request
Jul 13, 2021
See https://github.com/pulp/pulpcore/blob/master/docs/components.rst#distributed-tasking-system-tech-preview Remove use if pulpcore 'resource-manager' for pulp 3.14 pulpcore 3.14 replaces the 'rq' based tasking system with 'pulpcore-worker' So no longer need a service for 'resource-manager' on the docker-compose dev env tooling. Removed it from Makefile, the docker-compose.yml for each profile, and the 'start-resource-manager' invoked by the 'resource-manager' docker container. Required PR: #821 No-Issue
alikins
added a commit
to alikins/galaxy_ng
that referenced
this pull request
Jul 13, 2021
Required PR: ansible#821 No-Issue
chouseknecht
pushed a commit
that referenced
this pull request
Jul 14, 2021
Required PR: #821 No-Issue
awcrosby
added a commit
to awcrosby/galaxy_ng
that referenced
this pull request
Jul 20, 2021
….14.0 compat (ansible#821)" This reverts commit c491613.
awcrosby
added a commit
to awcrosby/galaxy_ng
that referenced
this pull request
Jul 20, 2021
….14.0 compat (ansible#821)" This reverts commit c491613.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Replace enqueue_with_reservation() with dispatch()
Primary difference is that enqueue_with_reservation()
returns an 'rq' job, and 'rq' support is being removed
in pulpcore.
The new 'dispatch()' returns a pulpcore.plugin.models.Task
object. dispatch() also no longer supports the 'options' arg.
pulpcore 3.13 removes 'enqueue_with_reservation'
https://docs.pulpproject.org/pulpcore/changes.html#id96
pulpcore 3.12.0 deprecated 'enqueue_with_reservation'
https://docs.pulpproject.org/pulpcore/changes.html#id152
See https://docs.pulpproject.org/pulpcore/plugins/api-reference/tasking.html#pulpcore.plugin.tasking.dispatch
for docs on dispatch().
More explanation of the diff between enqueue_with_reservation()
and dispatch() can be seen at:
pulp/pulpcore@753e4dd
and https://pulp.plan.io/issues/8496
Issue: AAH-706