Fix mirrored repositories for repos with distribution trees

[dralley]

Fix mirrored repositories for repos with distribution trees

Two problems:

• Mirrored repositories were creating one publication per
  sub-repository, which is in contrast to the way that pulp normally
  works. Standard publish puts the sub-repos into the same publication
  in a direct sub-directory because this is the only way to have
  everything distributed properly. Otherwise the other publications are
  in-accessible.
• Even once we fix that, we still have the issue that the original
  .treeinfo contains the original relative paths. So even for the
  mirrored case, we still have to rewrite the .treeinfo data to look at
  the new repos. Luckily it's not signed or checksummed so we aren't
  prevented from doing this.

closes: #9098
https://pulp.plan.io/issues/9098

[dralley]

Note that problem #2 is not fixed yet in this PR. @goosemania I'm curious if you have any thoughts about it.

Sadness all around. This has taught me how little I actually understand the mechanics of distribution trees, so I would appreciate more knowledgeable people checking my work here :(

[pulpbot]

Attached issue: https://pulp.plan.io/issues/9098

[goosemania]

@dralley, I agree with both problem statements. It's a pity I missed they obvious (to me) first one during code review, apologies for that.
I think the second problem is only for the centos8 case where the path to the AppStream is relative, e.g. ../../../../AppStream or something like that. For other repositories, I think no modification is needed, aka we need to fix it but the majority will work as is.
@dkliban is more knowledgeable in disttree metadata modifications, so he might see/remember more than I do.

[dralley]

I think the second problem is only for the centos8 case where the path to the AppStream is relative, e.g. ../../../../AppStream or something like that.

I assume that we would still need it to avoid clients hitting the original repos? Thinking about the disconnected use case, it doesn't matter if the upstream provided an absolute URL or not, because that repo is in-accessible.

Or do you mean relative backwards, rather than hosted in the same repo.

[goosemania]

I think the second problem is only for the centos8 case where the path to the AppStream is relative, e.g. ../../../../AppStream or something like that.

I assume that we would still need it to avoid clients hitting the original repos? Thinking about the disconnected use case, it doesn't matter if the upstream provided an absolute URL or not, because that repo is in-accessible.

Or do you mean relative backwards, rather than hosted in the same repo.

IIUC your last sentence, yes, the ones outside of a repo.

I was talking about paths in .treeinfo, they are always relative to the root of a repo, so you won't hit any original repos any way but the centos8 case is super weird and I think it's the main (if not the only) reason we have to rewrite the paths in treeinfo.
Let me try to show what I mean, I'm hoping not to confuse you.

I think we have all the options in our fixtures https://fixtures.pulpproject.org/rpm-distribution-tree/.treeinfo :

• main variant

[variant-Sea]
addons = Sea-Dolphin,Sea-Whale
id = Sea
name = Sea
packages = Packages
repository = .        <---- path within the repo, we do not need to rewrite anything
type = variant
uid = Sea

• alternative variant "normal"

[variant-Land]
id = Land
name = Land
packages = Packages       <---- this is actually a mistake in our fixtures I think :/ it should be `variants/land`
repository = variants/land  <---- path within the repo, we do not need to rewrite anything
type = variant
uid = Land

• alternative variant "abnormal aka centos8"

[variant-External]
id = External
name = External
packages = ../rpm-signed/   <---- this needs to be rewritten to the path below + /Packages
repository = ../rpm-signed/   <---- path is outside of the repo, we need to rewrite it because we publish any variant inside the repo
type = variant
uid = External

So in the majority of cases original paths to repository and packages will work as is, but in some cases they need to be rewritten:

• case 1, variant or addon are not within the repo path
• case 2, packages path doesn't have the pattern <repository_path>/Packages

From the Implementation point of view it's easier/cleaner to rewrite paths every time I think but these 2 cases are the reason why we are doing it.

[goosemania]

We agreed to fix problem 2 separately

[dralley]

Thank you @goosemania