Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

xbps-rindex: remove mode #76

Draft
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

Chocimier
Copy link
Member

No description provided.

@Duncaen
Copy link
Member

Duncaen commented Apr 11, 2019

I don't see how this solves the problem.
Removing packages from the index is not the problem of void-linux/void-packages#9745.
The problem is removing packages from the users installation.

@Chocimier
Copy link
Member Author

Removing current installations is already solved there. This together with void-linux/void-packages#10791 prevents new installs of empty packages, and also listing in xbps-query -R, https://voidlinux.org/packages etc.

@Duncaen
Copy link
Member

Duncaen commented Apr 11, 2019

I don't think its solved; it is a work around for something which would be better solved by xbps itself.

What I don't like about this proposed change in particular is, that it depends on a intermediate state of templates. This feels like something that is designed to just work with how current CI/build servers work.

I'd rather have xbps-checkvers output deleted packages and then have xbps-bulk or the future build server replacement handle this than xbps-src with pseudo bumps and a temporary keyword.

I guess I could accept a -R flag, which xbps-bulk and future build servers can use, but there are a few points:

  • It should work like the add method (get the repository from the path to the package file).
  • Simplify the unindex logic, I think you wouldn't need to compare versions and check reverts anymore.
  • But one big problem, which breaks the whole -R concept are noarch packages and how the current xbps-clean-repos script/cronjob works.
    The cleanup script explicitly adds all $arch and noarch packages to the repository before running xbps-rindex -r repo.
    I'm currently not sure why it does it for $arch packages, but for noarch packages this is necessary to have all the different indexes have the same (latest) version of a noarch package in the index, otherwise the wrong noarch package gets unlinked.

@Chocimier
Copy link
Member Author

I'd rather have xbps-checkvers output deleted packages and then have xbps-bulk or the future build server replacement handle this than xbps-src with pseudo bumps and a temporary keyword.

You convinced me, this feels better and solves point of @jnbr from void-linux/void-packages#10791.

for noarch packages this is necessary to have all the different indexes have the same (latest) version of a noarch package in the index, otherwise the wrong noarch package gets unlinked.

Removing binpkgs shared between indexes based on information from one index is a bad idea by itself. Can the logic be changed to listing binpkgs registered in each index, joining this list, then removing ones not needed by any? - Yes, that will not use xbps-rindex -r in its current form.

@Duncaen
Copy link
Member

Duncaen commented Apr 11, 2019

I always wanted xbps-rindex to open all repositories and to make i.e. the staging code aware of relations between different repositories, which currently still results in problems that the staging is solving for one repository.

I'm not sure what other problems this would introduce as it would have to lock all the repositories and then multiple builders might race when one of the builders is currently adding things.
Another idea was to keep xbps-rindex as it is and keep the xbps-src operations as simple as possible, but add something like xbps-repoadmin which would be a tool to merge and publish public repos, so instead of doing staging etc in the xbps-srcxbps-rindex xbps-repoadmin would selectively add things from the unpublished xbps-src repositories to public repositories.

I have to think more about this, xbps-rindex definitively needs to be improved.

@pullmoll
Copy link
Member

To avoid races between multiple builders trying to lock multiple *-repodata or *-stagedata files another global lock file like all-repodata.lock could solve this. It would be required only for the case(s) where more than the per arch repository is needed, e.g. when force adding a *.noarch.xbps to synchronize other architectures which also reference it.

@Chocimier
Copy link
Member Author

There is also Dijkstra's solution of ordering repodatas (say, alphabetically by path), then locking in that order and unlocking in opposite order.

@pullmoll
Copy link
Member

Ordering and trying to lock seems more complicated to me, though.
Of course xbps-rindex could always look for a all-repodata.lock and, even when not trying to acquire it, still wait for it to be released before trying to lock its own repodata.

@Chocimier
Copy link
Member Author

one big problem, which breaks the whole -R concept [...] cleanup script explicitly adds all $arch and noarch packages

This can be solved without xbps-repoadmin, by builbot removing and unregistering packages. List of packages will come from xbps-checkvers passed by xbps-bulk, as Duncaen proposed.

Will it be ok?

@Chocimier
Copy link
Member Author

  • It should work like the add method (get the repository from the path to the package file).
  • Simplify the unindex logic, I think you wouldn't need to compare versions and check reverts anymore.

Done

@Duncaen
Copy link
Member

Duncaen commented Feb 12, 2020

This would help a lot with quickly clean specific packages from the repositories while xbps-rindex is not aware of all repositories so we don't have to do the registering noarch stuff.

👍

@Chocimier Chocimier force-pushed the rindex-remove branch 3 times, most recently from 201dcc5 to 18e9ce6 Compare April 2, 2020 22:09
@Chocimier
Copy link
Member Author

Rebased.

@Chocimier Chocimier force-pushed the rindex-remove branch 4 times, most recently from 5010b7b to f1a1005 Compare April 15, 2020 18:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants