Add method to allow import of models with references to repo versions

[pulpbot]

Author: @gerrod3 (gerrod)

Redmine Issue: 8204, https://pulp.plan.io/issues/8204

---

Currently pulp import creates all the content models before creating the new repository version meaning that models with foreign keys to repository version can not be imported. This is currently the case with the AnsibleCollectionDeprecated model in pulp_ansible. It's possible other plugins have similarly difficult models to export/import that would need this solution.

[pulpbot]

From: @ggainey (ggainey)
Date: 2021-02-23T18:31:25Z

---

I'm not sure the realities of import/export allow us to do anything useful here.

'downstream' is not (ever) a copy of upstream. It's its own version of Pulp, whose admin can be doing anything they want to repositories (like adding and removing repo-versions). An upstream AnsibleCollectionDeprecated is deprecated for a specific repo-version; how do we map that to a downstream that may, for example, have never had a repo-version? Or one that has had 100, when upstream had 3? How do we expect the AnsibleCollectionDeprecatedModelResource to 'find' the repo-version it is deprecated for/against?

@gerrod @daviddavis, can either of you find any way for us to be able to make this possible? This may be a permanent restriction for this data-flow.

[pulpbot]

From: daviddavis (daviddavis)
Date: 2021-02-23T18:39:37Z

---

ggainey wrote:

I'm not sure the realities of import/export allow us to do anything useful here.

'downstream' is not (ever) a copy of upstream. It's its own version of Pulp, whose admin can be doing anything they want to repositories (like adding and removing repo-versions). An upstream AnsibleCollectionDeprecated is deprecated for a specific repo-version; how do we map that to a downstream that may, for example, have never had a repo-version? Or one that has had 100, when upstream had 3? How do we expect the AnsibleCollectionDeprecatedModelResource to 'find' the repo-version it is deprecated for/against?

We have a specific repo version upstream that has been exported and we're mapping it to the new repo version downstream that is created during import. This newly created repo version could be passed to the plugin.

@gerrod @daviddavis, can either of you find any way for us to be able to make this possible? This may be a permanent restriction for this data-flow.

One naive solution would be to simply dump a list of collections that should be deprecated. After the new repo version is imported, we create a AnsibleCollectionDeprecated for each collection in the list and attach it to the new repo version.

[pulpbot]

From: daviddavis (daviddavis)
Date: 2021-02-23T18:52:34Z

---

Here's maybe a bit better solution: during export AnsibleCollectionDeprecatedModelResource dumps the natural key for collections (namespace, name).

During import, the pulpcore code calls some list of model resources (POST_IMPORT_ORDER or something?) after creating the repo version and it sets a repo version property on these resources and then runs resource.import_data. The resource would have to define before_import_row to mung each row to account for repo version it has.

[pulpbot]

From: @ggainey (ggainey)
Date: 2021-02-23T20:13:05Z

---

OK, I see where my confusion was. Looking at the data model, I assumed that an AnsibleCollectionDeprecated must be associated with an arbitrarily-previous repo-version. The reality is, it is to be associated with, only, the repo-version that is being created at import-time.

Your proposal in #c5 is the direction I'm leaning. It needs some more think-time, but the general path (import post-RV-creation, BaseContentResource gets a resource-version-UUID field, subclasses use before_import_row to take advantage of that if desired) is I think the right direction to take.

Thanks for the clarification!

[stale]

This issue has been marked 'stale' due to lack of recent activity. If there is no further activity, the issue will be closed in another 30 days. Thank you for your contribution!

[stale]

This issue is no longer marked for closure.