New endpoint returning all collections unpaginated #488

fao89 · 2021-01-08T20:46:33Z

https://pulp.plan.io/issues/7940
closes #7940

pulpbot · 2021-01-08T20:57:03Z

Attached issue: https://pulp.plan.io/issues/7940

Attached issue: https://pulp.plan.io/issues/7941

bmbouter · 2021-01-11T20:36:36Z

CHANGES/7941.feature

@@ -0,0 +1,2 @@
+Introduce a new ``v3/metadata/<str:namespace>/<str:name>/versions/`` endpoint returning


Is this what it actually does? I though for 7941 it would list an unpaginated view of all CollectionVersions, but it would be v3/metadata/collection_versions/ not v3/metadata/<str:namespace>/<str:name>/versions/.

I was unsure about this endpoint,
if it was all collections versions for a given collection, or all collection versions.
So I coded the first approach, thinking I had to mimic what we already have but unpaginated.
I'll work on it and change the endpoint!

This changelog entry needs updating.

bmbouter

requested change to the implementation of metadata/collection_versions/.

pulp_ansible/app/galaxy/v3/serializers.py

pulp_ansible/app/galaxy/v3/views.py

fao89 · 2021-01-12T21:05:05Z

pulp_ansible/app/galaxy/v3/views.py

-        artifact = ContentArtifact.objects.get(content=instance)

        context = self.get_serializer_context()
-        context["content_artifact"] = artifact


@bmbouter CollectionVersionViewSet only presented artifact data on retrieve, so the strategy was getting the ContentArtifact and injecting it into the context.
For UnpaginatedCollectionVersionViewSet we need to present artifact data on list, so I changed the query to select related ContentArtifact

This is pretty slick. 👍

fao89 · 2021-01-12T21:14:32Z

pulp_ansible/app/galaxy/v3/views.py

+            serializer = CollectionVersionSerializer(page, many=True, context=context)
+            return self.get_paginated_response(serializer.data)
+
+        serializer = CollectionVersionSerializer(queryset, many=True, context=context)


the original endpoint for collections versions don't show all info we need for the unpaginated one:

{ "meta":{ "count":1 }, "links":{ "first":"/api/automation-hub/v3/collections/vmware/vmware_rest/versions/?limit=10&offset=0", "previous":null, "next":null, "last":"/api/automation-hub/v3/collections/vmware/vmware_rest/versions/?limit=10&offset=0" }, "data":[ { "version":"1.0.1", "href":"/api/automation-hub/v3/collections/vmware/vmware_rest/versions/1.0.1/", "created_at":"2020-10-09T23:10:13.142203Z", "updated_at":"2020-10-09T23:10:13.142234Z" } ] }

vs

[{ "version": "string", "href": "string", "created_at": "2019-08-24T14:15:22Z", "updated_at": "2019-08-24T14:15:22Z", "artifact": "string", "collection": { "id": "string", "name": "string", "href": "string" }, "download_url": "string", "name": "string", "namespace": { "name": "string" }, "metadata": { "authors": [], "contents": {}, "dependencies": {}, "description": "string", "documentation": "string", "homepage": "string", "issues": "string", "license": [], "repository": "string", "tags": [] } }, ..., { "version": "string", "href": "string", "created_at": "2019-08-24T14:15:22Z", "updated_at": "2019-08-24T14:15:22Z", "artifact": "string", "collection": { "id": "string", "name": "string", "href": "string" }, "download_url": "string", "name": "string", "namespace": { "name": "string" }, "metadata": { "authors": [], "contents": {}, "dependencies": {}, "description": "string", "documentation": "string", "homepage": "string", "issues": "string", "license": [], "repository": "string", "tags": [] } }]

So I use the CollectionVersionSerializer instead of CollectionVersionListSerializer, and CollectionVersionSerializer needs the ContentArtifact to get artifact data

bmbouter · 2021-01-27T20:53:01Z

pulp_ansible/app/galaxy/v3/serializers.py

@@ -182,15 +182,15 @@ def get_artifact(self, obj):
        """
        Get atrifact summary.
        """
-        return ArtifactRefSerializer(self.context["content_artifact"]).data
+        return ArtifactRefSerializer(obj.contentartifact_set.first()).data


Since this is replacing the removed line artifact = ContentArtifact.objects.get(content=instance) I think this could be

Suggested change

return ArtifactRefSerializer(obj.contentartifact_set.first()).data

return ArtifactRefSerializer(obj.contentartifact_set.get()).data

bmbouter · 2021-01-27T20:53:09Z

pulp_ansible/app/galaxy/v3/serializers.py


    def get_download_url(self, obj) -> str:
        """
        Get artifact download URL.
        """
        host = settings.ANSIBLE_CONTENT_HOSTNAME.strip("/")
        distro_base_path = self.context["path"]
-        filename_path = self.context["content_artifact"].relative_path.lstrip("/")
+        filename_path = obj.contentartifact_set.first().relative_path.lstrip("/")


Similar here to above ^

bmbouter · 2021-01-27T20:53:38Z

CHANGES/7940.feature

@@ -0,0 +1 @@
+Introduce a new ``v3/metadata/collections/`` endpoint returning all collections unpaginated


nitpick:

Suggested change

Introduce a new ``v3/metadata/collections/`` endpoint returning all collections unpaginated

Introduces a new ``v3/metadata/collections/`` endpoint returning all collections unpaginated.

bmbouter · 2021-01-27T20:54:01Z

CHANGES/7941.feature

+Introduce a new ``v3/metadata/collection_versions/`` endpoint returning all collections versions
+unpaginated


niptick:

Suggested change

Introduce a new ``v3/metadata/collection_versions/`` endpoint returning all collections versions

unpaginated

Introduces a new ``v3/metadata/collection_versions/`` endpoint returning all collections versions

unpaginated.

bmbouter · 2021-01-27T21:26:41Z

So I've been testing this PR and it's working great. I'm strugglng to remember why we made two endpoints though.... @fao89 do you remember?

https://pulp.plan.io/issues/7940 closes #7940

fao89 · 2021-01-27T21:39:53Z

So I've been testing this PR and it's working great. I'm strugglng to remember why we made two endpoints though.... @fao89 do you remember?

I moved the collection_versions endpoint to other PR: #509

fao89 · 2021-01-28T12:47:37Z

So I've been testing this PR and it's working great. I'm strugglng to remember why we made two endpoints though.... @fao89 do you remember?

I realize we need deprecation status, and collection_versions endpoint does not provide it

https://pulp.plan.io/issues/7941 closes #7941 (cherry picked from commit 0db524e)

bmbouter

Thank you!

fao89 requested a review from a team as a code owner January 8, 2021 20:46

fao89 force-pushed the 7940 branch from 2d6a94a to e32bff6 Compare January 8, 2021 20:56

newswangerd approved these changes Jan 11, 2021

View reviewed changes

bmbouter reviewed Jan 11, 2021

View reviewed changes

bmbouter requested changes Jan 11, 2021

View reviewed changes

fao89 force-pushed the 7940 branch 4 times, most recently from 86e0fa4 to cffd018 Compare January 11, 2021 23:03

fao89 requested a review from bmbouter January 12, 2021 13:08

bmbouter reviewed Jan 12, 2021

View reviewed changes

pulp_ansible/app/galaxy/v3/serializers.py Outdated Show resolved Hide resolved

bmbouter reviewed Jan 12, 2021

View reviewed changes

pulp_ansible/app/galaxy/v3/views.py Show resolved Hide resolved

fao89 commented Jan 12, 2021

View reviewed changes

fao89 force-pushed the 7940 branch from cffd018 to df5b36b Compare January 12, 2021 21:15

fao89 requested a review from bmbouter January 14, 2021 13:32

bmbouter reviewed Jan 27, 2021

View reviewed changes

fao89 force-pushed the 7940 branch from df5b36b to 9455ca7 Compare January 27, 2021 21:08

New endpoint returning all collections unpaginated

a9ac759

https://pulp.plan.io/issues/7940 closes #7940

fao89 force-pushed the 7940 branch from 9455ca7 to a9ac759 Compare January 27, 2021 21:39

New endpoint - all collection versions unpaginated

25ce25b

https://pulp.plan.io/issues/7941 closes #7941 (cherry picked from commit 0db524e)

fao89 requested a review from bmbouter January 28, 2021 17:31

bmbouter approved these changes Jan 28, 2021

View reviewed changes

bmbouter merged commit 6be64b8 into pulp:master Jan 28, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New endpoint returning all collections unpaginated #488

New endpoint returning all collections unpaginated #488

fao89 commented Jan 8, 2021

pulpbot commented Jan 8, 2021

bmbouter Jan 11, 2021

fao89 Jan 11, 2021

bmbouter Jan 11, 2021

bmbouter Jan 12, 2021

bmbouter left a comment

fao89 Jan 12, 2021

bmbouter Jan 27, 2021

fao89 Jan 12, 2021

bmbouter Jan 27, 2021

bmbouter Jan 27, 2021

bmbouter Jan 27, 2021

bmbouter Jan 27, 2021

bmbouter commented Jan 27, 2021

fao89 commented Jan 27, 2021

fao89 commented Jan 28, 2021

bmbouter left a comment

		@@ -0,0 +1,2 @@
		Introduce a new ``v3/metadata/<str:namespace>/<str:name>/versions/`` endpoint returning

	return ArtifactRefSerializer(obj.contentartifact_set.first()).data
	return ArtifactRefSerializer(obj.contentartifact_set.get()).data

		@@ -0,0 +1 @@
		Introduce a new ``v3/metadata/collections/`` endpoint returning all collections unpaginated

	Introduce a new ``v3/metadata/collections/`` endpoint returning all collections unpaginated
	Introduces a new ``v3/metadata/collections/`` endpoint returning all collections unpaginated.

		Introduce a new ``v3/metadata/collection_versions/`` endpoint returning all collections versions
		unpaginated

New endpoint returning all collections unpaginated #488

New endpoint returning all collections unpaginated #488

Conversation

fao89 commented Jan 8, 2021

pulpbot commented Jan 8, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bmbouter left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bmbouter commented Jan 27, 2021

fao89 commented Jan 27, 2021

fao89 commented Jan 28, 2021

bmbouter left a comment

Choose a reason for hiding this comment