PULP && reuploading of a duplicate NEVRA problem

[praiskup]

After a f2f discussion with the PULP team today, it's clear that duplicate NEVRAs - that we allow our users to produce - are going to be a problem; we are touching on a fundamental design decision in PULP: one repository version may never point at duplicate NEVRAs.

Situation:

1. we upload foo-1.1-1.noarch.rpm from a build 0001 -> we get repo version 1
2. we upload foo-1.1-1.noarch.rpm from a build 0002 -> we get a repo version 2, where the NEVRA is updated (just a different file checksum)
3. we can continue with repo version 3 from build 0003, etc. ... but

The latest one wins, as the duplicate NEVRAs are being (re-)added. Always. Then we can rollback the repo versions (if the previous repo versions are not garbage collected), but repo versions is something we probably don't want to track in Copr, it's pretty much impossible with the builder's concurrency.

The thing is we want to allow our users to remove any of the duplicate builds. Then, when a user does so -> Copr can say "hey PULP, remove this NEVRA from that repository" (we can refer the artifact by NEVRA, hashsum or href), but the result will be that the NEVRA is going to be removed without a replacement, so....

4. we say remove foo-1.1-1.noarch.rpm, we get repo version 4, artifact from build 0003 is removed, but there's no foo package at all (the build from 0002 is not going to be re-added).

We should decide whether this is a problem, how big the problem is, and how much we want to push to find solutions or workarounds to this.

[praiskup]

The team agreed that this should be an OK change if properly discussed with the fedora-devel list first, and well documented.

Technically we agreed we need to upload duplicates, but what happens with old artifacts seems less important.
Especially if we can somehow warn the user there's no way back to the previous duplicate NEVRAs (PULP API warning, that we could propagate to the end-user).

[FrostyX]

@praiskup to write an email to fedora-devel

[praiskup]

Sent here: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/ELCAQMCBOJOK2DQL5YEJH4RKGBMGYUFB/

[penguinpee]

From your e-mail:

But on the other hand this also means that, when the latest NEVRA build is removed, Copr will drop the NEVRA from the metadata (it will not present the older builds).

Shouldn't (automatically) or couldn't (manually) that be resolved by regenerating the repository?

[praiskup]

Shouldn't (automatically) or couldn't (manually) that be resolved by regenerating the repository?

With pulp, "re-generating" is probably not going to exist. We are not yet sure. The problem here very much depends on whether we will be able to keep the duplicate RPMs hosted in PULP (and not garbage collect them). If we are able to keep them (without too much hassle), we should be able to re-promote them upon user's request.

[praiskup]

Closing as a resolved question, but feel free to continue the discussion.