shallow rez-cp can corrupt package if there are overlapping variants

[nerdvegas]

Consider a package with 2 variants, with subpaths 'A' and 'A/B'. A shallow copy can corrupt the source package because of the following:

• Variant <newpkg>/A is created as symlink to <oldpkg>/A;
• Variant <newpkg>/A/B is created as symlink to <oldpkg>/A/B;
• Whoops, this actually has created a self-referencing symlink at <oldpkg>/A/B.

Overlapping variants (where one variant subpath is a parent dir of another subpath) is inherently problematic for shallow package copies.

Solution: Don't symlink at the variant dir level; instead, symlink everything inside the variant dir (excepting dirs that may be parent of overlapped variant).

Note that when dict-based variants exist, this problem will go away (as will various other problems related to current limitation of variants). For dict-based variants, I think we'll stick to the variant dir symlinking, since it's simpler.

[bpabel]

Do you think that symlinking between versions, and creating dependencies between versions might be a bad idea? It means you can't simply clear out old versions because now there's interaction between the versions.

[nerdvegas]

Use at your own risk. We have a valid use case here at Method. I'm not really a fan of symlinking variant payloads myself, which is why this is optional. We have a set of package releases which always exist - so called "raw" packages. We are then creating other package repos which are a sort of "view" into these packages, and it's these which are symlinked to the raw package. Any package archival activity ensures that a raw package is never deleted unless all "view" packages to it already are. Hth A

…

On Sun, Feb 3, 2019 at 2:33 PM Brendan Abel ***@***.***> wrote: Do you think that symlinking between versions, and creating dependencies between versions might be a bad idea? It means you can't simply clear out old versions because now there's interaction between the versions. — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#563 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABjqSoyHDqyqSEuiy7mevkQeiaOeI2Ezks5vJlhxgaJpZM4aCOf1> .

[bpabel]

Ah, thanks for the explanation. Hadn't thought of a package repo setup like that.