Growing lack of package maintainers #6584
Comments
|
I personally think your last sentence is what's causing quite a bit of extra workload and makes PRs some more or less come to a grinding halt which in turn makes contributors give up/stop caring. https://www.freebsd.org/doc/en/books/porters-handbook/makefile-maintainer.html Create a generic template on the layout of a Makefile which you should follow in general https --> http --> ftp |
|
also, something like this should be taken into consideration In addition there should be something like TravisCI that actually spits out full logs or some other kind of auto testing facility in place and I think a "package manager team" should also have the right to refuse a port if it's for a very limited amount of users (specific usage). |
|
@thess, I do prefer to accept only packages with maintainers. However, in fact, it is mostly the same situation of a package that was once maintained and became orphan. If we keep orphan packages, we might accept unmaintained new packages as well (it's just an analogy, not my desire). If a package is unmaintained, should it be removed automatically just after branching a new major release? Or should it only be removed when a major issue appear, like a broken build, security vulnerability or it simply does not work anymore? (in fact, this applies to any package, "with a name in MAINTAINER" or not). It would be nice if a bot, after some time, could open build and security issues (mentioned in a CVE) automatically and, after some more time, open a remove PR (move to packages-abandoned). FreeBSD ports is the same situation of openwrt-packages. Those rules mostly fit our needs. That's a good start. We do lack a little more explicit rules. For example, I do have commit access and I can commit new packages. However, I'm not sure if I must do it. I use my rights only for "maintaining my packages", not bothering others with updates/minor commits. What is the process of accepting new packages? We do already have a team of "package maintainers" in github. We could explicitly have a new "package manager" team that rules whether to accept new packages, reset maintainership or remove packages with major issues. |
|
@thess |
|
@dibdot |
|
A more pragmatic approach would be to keep the packages that still compile, until they break and no new maintainer is found? |
|
I would really love to have usage statistics (send installed packages + version) and / or a list of active user for each packages so it's easy to contact other people that do care about the package and know there usage |
|
@Andy2244 - I agree with keeping the packages and when they break , mark them BROKEN (no longer will build). How we deal with PRs and new package submissions without a MAINTAINER, needs to be addressed too. @champtar - If you are looking for built package downloads, we are attempting to get those stats. There is a WIP, for example, here: https://downloads.lede-project.org/stats. Also, there is no hard/fast rule preventing others from adding themselves to the PKG_MAINTAINER var. Perhaps we could encourage that. |
|
I noticed that sometimes people only assume maintainership for the sole reason of getting the package back into the binary repositories because they have an urgent need for it. I can also understand that not everybody wants to enter a multi year maintainer commitment only to have a specific package available. I think there are multiple things that can be done to partly address the problem of unmaintained packages or the lack of manpower in general:
|
|
If security is of focus the keep package until it doesn't build argument will in many cases do the exact opposite. |
Well when someone uses IB, we can track the downloads of individual packages, right? |
|
Also, (I've been away a while) are there folks contributing regularly that maybe should have commit access? |
|
I'm wondering if would help (I could help set it up) to have something like Ubuntu PPA's or Fedora COPR, where users can easily build packages they (and other interested folks) can use on their devices? I don't think for those repos it'd be required to build for every arch. Thoughts? |
|
I have ~25 pull requests with most of them having inactive maintainers. I've thought about this for a bit. Maybe something like arch is needed where you have maintainers but you also have Trusted Users that do the grunt work regardless of the maintainers. Arch Linux has 13039 packages. ~250 are out of date. That's 2%. OpenWrt has less packages and more of them are out of date. On throwing packages in abandoned repo: Unless hard drive space is an issue, I completely disagree. Just because there's no user today does not mean there won't be one tomorrow. I am however a fan of doing so when a better replacement is found. I have two pull requests that replace libjpeg and pkg-reconf with -turbo and pkgconf. I should probably update the latter. On build failures: I have no idea how to read buildbot failures. An example: fdm does not compile on arm_mpcore: https://downloads.openwrt.org/snapshots/faillogs/arm_mpcore/packages/fdm/ . The logs seem fairly useless. Or I have no idea how to read them. Many of my pull requests replace using git with using a tarball generated by github. This has the benefit of reducing hash mismatches between different hosts. There was a situation once where one of @wongsyrone 's version bumps had a different hash than what was generated locally for me from the git repo. Won't mean anything unless merged though... Finally, since we have @sdwalker 's lovely uscan tool, might as well fix up packages that show issues: https://sdwalker.github.io/uscan/ |
Which one? |
|
edit: I believe I double checked on my end at the time as well. In any case it's irrelevant now. |
|
Rosen Penev <notifications@github.com> wrote:
I have ~25 pull requests with most of them having inactive
maintainers.
And the vast majority of them are changing packaging for marginal
gain. Inactive is also relative. For a maintainer of a package
that updates once a year, maybe less, being "inactive" just
because they're not approving changes withing a day or two, or
even a week, isn't necessarily "inactive"
I've thought about this for a bit. Maybe something like arch is
needed where you have maintainers but you also have Trusted
Users that do the grunt work regardless of the maintainers.
Arch Linux has 13039 packages. ~250 are out of date. That's 2%.
OpenWrt has less packages and more of them are out of date.
And arch users famously complain that things don't work. Updated
and compiling isn't always the same thing as functional. I'm not
saying we shouldn't update things, but updating every package in
openwrt the day upstream releases something isn't necessarily any
sort of sign of quality improvements. Particularly given the
nature of openwrt targets, it often exposes breakages.
Are there any realllll problems here? I've personally felt that
the new packages repo has been wonderful and largely welloiled
compared to the old system. There's always room for improvements,
and some packages definitely warrant multiple maintainers, but
this all feels like a storm in a teacup.
Cheers,
Karl P
|
|
In addition to disappearing maintainers, we also have a challenge of low activity by committers to handle pull requests. There are only a few persons who actively merge PRs on larger scale. Quite many committers in the package feed repo seem to only care for a the few packages that they are maintaining by themselves. |
|
Hannu Nyman <notifications@github.com> wrote:
In addition to disappearing maintainers, we also have a
challenge of low activity by committers to handle pull
requests. There are only a few persons who actively merge PRs
on larger scale. Quite many committers in the package feed repo
seem to only care for a the few packages that they are
maintaining by themselves.
So give more maintainers commit rights. I was given commit rights
in the understanding that it was to streamline the maintennance
of my own packages, and I was being trusted to not step on other
peoples work.
Are other maintainers _not_ given commit rights? Why not?
Cheers,
Karl P
|
|
I maintain 4 currently. I have none. |
I'm not sure that's their motivation. Some of them might be hesitant to step on someone else's toes, hence they only work in their own sandbox. I'll very occasionally commit in another package if (a) the commit looks very safe, (b) it's been discussed adequately, and (c) the actual maintainer has had adequate time to speak up but hasn't... |
|
@pprindeville That's a good point; I think the 'owned' packages sentiment tends to that feeling, which (I think) is why @jow- and others have suggested a more general 'packages feed' maintainer over all packages being owned (with uneven levels of participation (for various reasons)) by an individual maintainer or (rarely) team. |
|
@jow- @dibdot @thess my bias tends to agree with your statements. I would think part of the problem is "packages-abandoned" is not much of an option. The choice is regular package or dark abyss of doom. For new packages someone would like to dump in OpenWrt without a maintainer, then maybe "packages-experimental" or "-unstable" or "-bleedingedge" is appropriate. There could be a process to get consensus to move to regular packages after time, use, and a responsible maintainer. Now thinking out loud, more than making any specific recommendation ... Fringe packages could be retroactively shoveled to experimental if (1) they are still apparently relevant in the wider world, but (2) they lack a maintainer or has been absent too long. This repo then could even have its own menuconfig category. This menu rule is one of few strict enforcement in "packages-experimental" and the other is Travis-CI passes. That way it can be part of a proper release, and binaries are available, but amateurs have fair warning of its quality control. But wait, there is more. Experimental can be a good place to train or test new committers before letting them have at regular packages. Will they help others? Will they "be nice to each other?" Presumably experimental is a place with a lot of churn and new widgets (which may die, but fun in the mean time). |
|
@thess volunteers for triaging (once process is defined) |
|
@EricLuehrsen I have some ideas about using Travis-Ci's deployment option to create a PPA / COPR type option for OpenWrt. The other piece of course is the deployment 'target' (e.g. something that publishes the built packages - that is is willing to accept them from Travis-CI). It's a significant chunk of work, but it's doable. This RUPA (Random User Package Archive) system would be partly a Travis chunk that would be versioned and used by user's in their own public GitHub repos, and published to the RUPA server via credentials one signs up to get. A final piece is an luci-app and/or cli script so users can add the appropriate key to their opkg keys, more easily (but with the usual warnings about using packages from random archives). My though with this is to reduce how much hosting and effort the OpenWrt project has to do wrt third party packages. I do like the idea of experimental, especially as a comitter testing ground. In any case I think a strategy change is needed. |
|
As I said in #6748: I would like to say that impatience (and I'm badly guilty of this too), is a big part of the problem here and with the perceived maintainer problem with the packages feed, and/or the pushing of commits by non-maintainers (that turn out to be problematic). |
|
I wonder if we should consider current packages as -universe (since that's largely what already is) and maybe have a new repo (packages-standard or packages-coreplus or packages-curated) or something that indicates that is closer to core strictness and nitpickiness and/or explicit active maintainer care (would still need e.g. 3 month rule on maintainership) where it's expected the maintainer has to okay commits, unless there is some security urgency and it's been over a month and it's been tested by at least two people. The current/regular packages feed could included -community versions of the same packages or somesuch that get the bleeding edge and/or regular -universe treatment. |
|
I'm with @diizzyy on this. I see no value is splitting packages in separate repositories. |
@jow- That seems too easy. I think we need some sort of quorum (or at least a threshold of N people for N >= 3) so that a single wild hare doesn't end up taking OpenWrt off in some random direction that the rest of us aren't in agreement with. There have been increasing efforts to make OpenWrt routers into DLNA servers as well, and to the extent that this doesn't compromise security I'm not going to object. But there will come a moment where the camel's back gets broken...
14 days seems a bit brief. It took me 3.5 weeks to figure out how to get Perl building again when we went from 5.26.2 to 5.28.0 because of some upstream weirdness that didn't account for cross-compilation. If someone shows that they're actively working on a problem and explaining what efforts they've made, I'm willing to continue to extend them grace for quite a bit longer than 14 days... |
@Andy2244 Because we typically review the packaging, i.e. the What happens if it's a well-known package, but it's not coming from the official repository or download server, but is instead a modified set of sources? |
|
My biggest concern with splitting/shuffling packages around is the lack of manpower, we struggle to keep up with reviewing and merging PRs in a timely manner and I'm not very confident in adding more housekeeping chores will be beneficial. IMHO it would be better if we can come up with proper packaging documentation along with guidelines (deadlines, dos and don'ts etc) and once that workflow works we can add more housekeeping... |
|
@neheb That's because you and @diizzyy happy with packages as -universe (as it basically currently is) - some folks want more stable and curated packages, and you can't keep both sets happy with the same set of rules. I think @chris5560 got fed up with the impatience and lack of QA, and there are probably others like @thess who I'm guessing would like a more solid packages feed than -universe feed. |
|
@diizzyy I'm not talking about splitting, but starting fresh with a whole new (much smaller) repo for those who want a more stringent feed with more emphasis on individual maintainers, who work at a slower pace than seems to be wanted by the bleeding edge -universe repo we have. I'm thinking of the current packages kind of like a a cross between -universe and debian's -testing or -unstable, and the new feed being more like centos 7. I think @poranje might be another who'd prefer a less bleeding edge packages feed. Maybe @karp too, and maybe @wongsyrone (but those are wild guesses). |
|
Exactly what does that mean? http://howfuckedismydistro.com/debian/ ? :-) |
|
@diizzyy the release branch seems to me to be viewed as set in stone except for extremely important fixes (which is a little different than having a slow-moving and stable but still getting new stuff, and more bug fixes than say debian -stable feed). |
|
@diizzyy I'd rather use Debian for a server than say Arch....just sayin... |
@cshoredaniel We'd need a dependency graph so that nothing in the stringent repo required anything that wasn't... Just like for base. |
|
@pprindeville Yes, that would be a requirement of the new repo is that it would only depend on core and itself. |
|
All of this is starting to sound like CentOS vs. EPEL... |
|
@cshoredaniel |
|
@diizzyy For the 'curated' repo it wouldn't likely be folks like you and @neheb who want bleeding edge but individual maintainers who are active in that they will fix and work on issues, just on on necessarily on a time frame that would satisfy the impatient / bleeding edge types. Part of the problem I see for maintainers who care about quality but don't have oodles of spare time, is that they feel trampled over by folks who push stuff with hardly any (if any) testing. It's not like -universe where you have to keep up with multitudes of packages you as a maintainer don't really care about. OTOH who's going to work on -universe if most folks who work on individual packages don't want to become 'at-large' and/or aren't interested in dealing with bleeding edge demands? And who would want to anyway? I certainly am not interested in dealing with -universe except to keep a bleeding edge version of the couple packages I maintain (it would however be a good way to have the development version of NUT which has something like a two to three year release cycle, availaable). But if there is problem finding folks who want to work on -universe, maybe -universe laxity rules are a problem? |
|
The package repo has pretty much no security/vuln awareness at all and most packages that we've touched have been very outdated. Are you suggesting that it's a better course of action to not upstream to the repo and happily ignore those issues? We're fully aware that new releases may introduce new regressions but seeing that there's no secteam (which Debian etc has) the best course is to follow upstream and report issues which seems to be agreed upon as PRs gets merged. There's no way one could test packages on all platforms and in all scenarios, that's just unreasonable but feel free to elaborate on how to accomplish that. Most maintainers/contributors have one or perhaps two different platforms they can test on, the rest is best effort. I don't mind splitting into 500 repos/branches or whatever but I'm asking if it's realistically possible as I'd hate to see peoples effort go to waste just because someone wants to pull off something grand that isn't going to fly in the first place. |
|
+1. AFAIK I'm the only one that has added PKG_CPE_ID to this repo. Every applicable package in the main repo has it but the other feeds do not. Not to mention uscan currently lists 42 CVEs. |
|
That's part of my point, really. The repo is too big and most people aren't interested in with tons of packages that don't know. That why I suggest a 'curated' (smaller) repo and this one that folks (like you) who are interested in doing treewide updates and such on packages that are released to the wild for testing, rather than through much (if any) personal QA first, can take care of. For those of use that want a few well-maintained packages that don't get tromped over by impatience, we get it, and for those that want everything and the kitchen sink that stays as longs at builds; well there is that too. What I am opposed to is continuing course with the status quo -universe and no 'curated' repo because I don't think the proposed rules will bring things up the quality levels I at least am looking for. |
Something to add to the "commit instructions checklist" as a suggestion if missing. Please, don't slit packages. Don't do the oldpackages->packages once more. openwrt/packages has, by conception, more flexible security constraints. Just check the number of committers. If a user is really concern about outdated packages and malicious commits, openwrt/packages should be used with care (or simply stick with openwrt/core). What is the idea with the curated/universe repos? Enable both by default? It will not change anything for most users. Disable universe by default? Most users will rant. Those users that know how to enable/disable repos are technically able to evaluate packages source by themselves. Adding a Luci interface will not help a lot, as the user still needs to know that the division exists, their difference and how to enable the universe repo. It will significantly increase the complexity. I'm against the introduction of any code to a binary release without a human intervention, as it would allow malicious code easily get into user machine. We could, although, after autochecks and objection period pass, accept commits to an already existing package if it comes from its maintainer. In fact, there isn't much more confidence in committers than in maintainers. When openwrt/package was created, I guess that any maintainer that asked for commit access got it. A new publish CVE is potentially equivalent to "committing malicious code". The difference is that the code was already there. So, if we are against "automatically committing new code", we should also "automatically disable known insecure, potentially exploitable code" until someone tells it is secure. We can classify a package into some states: maintained, insecure (have known CVE), outdated (have known new stable release). uscan is already doing the job. I would prefer to not touch Makefile in order to avoid a unnecessary build. It would be nice to have a bot (uscan-bot?) that created issues whenever a new release or CVE appears. If we standardize commit messages related to version bump or patching CVE, the bot could even close the issue by itself, even not mentioning the issue number directly with github dialect. We could use these issues to track the package state (instead of modifying Makefile). If there is an open "security issue" for package foo, opened by the uscan-bot or labeled as "security issue" by a committer or the maintainer for more than X days, consider package foo as insecure and avoid shipping its binary in a new release. After an human evaluation (preferably by the maintainer), a committer could close this issue, with comments, explicitly ignoring the open security issue. The same goes for outdated packages, but they might still be shipped even with open non-security issues. One problem is that we does not have a direct map between maintainer and github user. It would help if we could add an @< github-user > to PKG_MAINTAINER. If OpenWrt core does not want it, we could introduce a new file for that inside package directory (pkg1/.github-maintainer?). The guidelines @jow- mentioned guidelines for a package maintainer. I was thinking about something special for a package reviewer limited to those things that still need a "natural intelligence" to check:
This could be used for new packages, even when they comes from a user with commit access, specially for the "objection period". |
|
Since the notion of a smaller 'curated' repo is unpopular, how about, in keeping with what @jow- said in terms of not splitting things, a new menuconfig section and tag e.g. PKG_APPROVAL:=maintainer or something like that that indicates that changes to the packages require maintainer approval (with reasonable guidelines about what constitutes 'active' so that if a maintainer disappears that packages don't get 'stuck', even if the maintainer opted for this requirement) except for CVE's or build failures affecting more than just that package (or the PKG_PRIORITY:=critical)? |
|
Oh, and can we please define a process for dropping packages: like getting whowever runs uscan to actually submit issues about CVE's not just list them in uscan (for which I can never remember URL anyway). Also uscan should have a feedback mechanism so that if the CVE is not relevant in the OpenWrt package this can be noted (for example occurs in combination with Mozilla NSS but NSS isn't even in packages repo). |
|
Very interesting what @luizluca proposes. Following up on that, another idea. As already stated by @diizzyy, OpenWrt by lack of a security team, often has no much better choice than to follow upstream, also because presumably the contents of upstream sources are mostly rarely, if ever, checked for malicious code. Would tapping onto the acceptance of updates by renowned distributions like Debian or Redhat/CentOS of package's upstream source updates be an option ? In that case even creating and merging such updates automatically (when other automatic tests are passed) might to some extend be an option. To some extend because security commits are often released together with other changes and are for that reason back-ported onto stable versions or, other way around, up-streamed, by distributions like Debian. Do not really know what would be possible or not, and how to automate this. Also the idea the that maintainer and reviewer could be distinguished might help to attract more interested people that could help out. It already is possible to add review remarks, anybody can, but many people do not want to be nosy or intruding. Maybe, a separate review, non-committer/maintainer role, lowers the bar. |
|
@cshoredaniel can we please not automatically label things based on arbitrary existance of magical "CVEs" ? They're not bulletproof, and they're not all fatal flaws, and sometimes they're not even valid. Having the filed CVE information available is nice, but auto hiding or worse, removing packages based on them seems wildly authoritarian. |
|
@karlp hence my comments about ways to address the uscan 'CVE' notation. I didn't mean no human intervention hiding, I meant that the CVE's should result in say an 'issue' to be examined, and if the CVE is valid the package is marked I did say
|
|
I want to apologize for my attempt to guess other people's opinions based on past statements. It's a bad habit of mine that I'm trying to rectify. It's not much of a defence but it was more by way of trying to encourage said individuals to express their actual opinion, even if it didn't come across that way. |
|
OK folks - thanks for all this (mostly) useful feedback. I am going to close this topic now as it has been pretty talked out. For the next step, I propose we draft a document outlining a SIMPLE review process/timeline for handling PRs. Topics covering what happens if no-one responds and the possible auto-merging or closing of neglected PRs should be included. Also, noting the difference of handling new packages vs updates to existing. If no one volunteers to draft such a document, I will make a rough outline next week. Create a PR for a new document: REVIEW.md - Markdown preferred -- Or some other appropriate title. |
It seems we are collecting a number of packages with no MAINTAINER lately. This was not a direction chosen when the new Github repository was created and we crafted guidelines. The purpose was pretty clear that we wanted individuals responsible (OK for more than 1) for the submission of and updates to each package. The main goal at the time was to cleanup the old repository given that a number of packages were outdated, unused and/or didn't build at all. I do not wish to see this repository to degrade to that state.
That said, If there is enough consensus among the members, to drop this requirement, I will gladly go ahead and approve/merge updates without maintainers as long as folks sign-off as having tested the change and one other person has ACK'd the PR. Personally, as a package maintainer, Likewise, for my packages and dependencies, I take the time to at least build and maybe test package changes that are not always straightforward.
I am also looking for volunteers, to help review and possibly close old PR's (and issues?) given some criteria and process TBD.
The text was updated successfully, but these errors were encountered: