Group "related" repositories, allow hiding "obsolete" repos

[fingolfin]

I am sorry if this is a duplicate, but I looked through the list and couldn't find anything that seemed similar. But here is an idea for improvement ...

So I like to use repology to see for a package I maintain (gap, in case you wonder) which distros ship it, and which have its latest version.

Unsurprisingly, if e.g. CentOS 8 doesn't have it, then neither do CentOS 7 nor CentOS 6. The same goes for old versions of Debian, Ubuntu, Fedora, and many more repository "families".

For this reason, and likely more, I think it would be nice if "related" repositories could be grouped in the UI, with an option to hide all but the most recent in a group. So that on a page like https://repology.org/project/gap/versions, instead of seven Fedora variants, there is just one entry "Fedora", which lists the most recent version of the package. How exactly to do this: I guess there are many ways to do it, depending on how much time and effort one might be willing to spend on it. A very simple one might be to have two versions of the page, one as it is now, and one where all "families" are merged into one. One can vary what exactly "merged" means:

1. one could require a strict linear order on the repos in a family, and only show the latest; that would work well in many cases, e.g. Fedora and Debian; but it might not in some others?
2. just show the latest version found in any of the group repositories (difference to 1 might be if a package is dropped from a family; so if Debian X has the package and Debian X+1 does not, then with 1, the package is not shown, with 2. the version in Debian X is shown)

More fancy would be to have a single page, with families "folded" together, and an option to unfold a family so reveal all its members. And some widget to fold / unfold all (your choice which state should be default ;-))

Now, I realize this is not a wishing well, and you may not be interested in doing this, but I thought I might mention it. I'd offer to help with it, but I am afraid I have little time right now (busy with teaching, doing all of it online-only adds a lot of extra work for now, at least until all the processes have smoothened out).

Thank you for making repology.org!

[AMDmi3]

I don't plan to implement this, for it would provide misleading overaggregated information. For instance, it would report that gap is at 4.11 in Debian and Fedora, while in fact no official releases have this version - it's only in unreleased Fedora 32 and rolling Rawhide, and Debian Unstable not much people use in production. Excluding development repos from aggregation would not solve the problem, as there are still (most) people who use non-latest or LTS releases, and it would additionally require extra effort of keeping repository status (e.g. tracking distro releases) with minimal lag. In other words, regardless of how you do it, it would essentially report "some" random version, not related to what most users actually see in their package managers. So I'm going to keep it simple and report the true status in each repository separately.

Some more related thoughts: repology/repology-updater#895 (comment)

PS. Some repos use NrNpN versions for gap, is this some kind of old schema? Should I normalize it to N.N.N?
PPS. I see you're Fink contributor, you might be interested in helping supporting it in Repology (repology/repology-updater#243)

[fingolfin]

OK, fair enough (even though sad for me, but OK :-).

Re PS: short answer: GAP always used X.Y.Z versions; the Debian maintainer misunderstood something, and thought he should use XrYpZ; then tons of packages are based on the Debian package. Long answer: this is due to a misunderstanding: when GAP was created in the 80ies, development was done on Atari ST, which used a DOS compatible file system, and thus had 8.3 filenames. So the GAp release archives could only contain one dot, and used filenames like gap3r4p4.zoo. Then, as so often happened, later on nobody questioned it and it stuck with that format until I convinced people in 2018 that we could now use fancy symbols like . and - in filenames, and even use more than 8 letters in the basename ;-)

PPS: Yeah, I even used to be project lead for Fink, like, a decade ago... but it's fading away... I thought about this, but honestly, I can't be bothered (I don't even package GAP, which is now my main open source work, for Fink; go figure). That said, I'll relay this, maybe somebody else on the Fink team is interested.

[AMDmi3]

OK, fair enough (even though sad for me, but OK :-).

Out of curiosity, what would be your use case for the aggregated version? Maybe I am missing something.

GAP always used X.Y.Z versions

Oh I see now that XrYpZ is only used in Debian & derivatives and with 4.11.0 they have switched to X.Y.Z which is ordered correctly, so it looks like no action is required here.

That said, I'll relay this, maybe somebody else on the Fink team is interested.

Great, thanks!

[fingolfin]

Well, I just like to use this information to keep an eye out for each distribution, and then I engage packagers if I see their packages are lagging behind. I often work with them to get it update For this, I really am only interested in the latest version they package anywhere; it doesn't matter if Debian Unstable, Stable and Experimental are mixed up.

Granted, for GAP there are not too many distros which package it (although I am working on changing that), but I imagine other package authors might have a similar use case. But maybe I am also unique with this. Don't worry, I'll cope ;-).

I guess if I had any way to selectively filter out which repositories are (not) shown, then this would be sufficient for my purposes.

As to the GAP versions: Well, if it was possible to filter the versions and convert 4r6p5 to 4.6.5, that would remove some clutter from the "Versions" list on https://repology.org/project/gap/information which otherwise likely would stay for long time. But it's not important, you have bigger fish to fry.

(BTW it's ludicrous that there are packages of GAP 4.3.5 and 4.4.12 out there; clearly these have been unmaintained for over a decade. No luck in contacting anybody there so far. Anyway, w/o repology I wouldn't know about these at all. I really appreciate you work!)

[AMDmi3]

Well, I just like to use this information to keep an eye out for each distribution, and then I engage packagers if I see their packages are lagging behind. I often work with them to get it update For this, I really am only interested in the latest version they package anywhere; it doesn't matter if Debian Unstable, Stable and Experimental are mixed up.

This sounds valid in fact. I also haven't realized at first that you need not a badge but a page which is not if fact as abusable.

I can think of different approach for this purpose - a maintainer-centric view, e.g. a list of latest versions per maintainer. It would have a positive property of merging much more derivatives (e.g. all repositories https://repology.org/maintainer/ballombe%40debian.org into single maintainer entry), but on the downside it won't be able to handle repos which do not have maintainers or do not provide maintainer information, would generate. May go combined way as well, e.g. maintainers if available, otherwise repository families, but here it does inconsistent and overcomplicated. May depend on repology-updater#917 too.

I'll leave the issue open to think on it more some time, however consider it wontfix for now.

PS. I've normalized XrYpZ versions for consistency.