New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

What to do with abandonned/unmaintained/orphaned projects? #1506

Open
guyzmo opened this Issue Nov 29, 2016 · 28 comments

Comments

Projects
None yet
@guyzmo

guyzmo commented Nov 29, 2016

Hi,

I'm posting this issue following an IRC discussion about what would be the policy in case a maintainer of a project is not around anymore (in case they left to become a shepherd in the mountains far from all technology… or when they die, because that happens too).

To quote @dstufft on IRC, that's what the current policy is:

We have a fairly adhoc process that skews heavily towards "if we can't get permission from the original maintainer then nothing happens"

Report a project as orphan?

As a developer and direct user of Pypi's warehouse, I'd be happy to be notified when a given published project is orphan (or likely to be orphan). So it'd be great to have the ability to report that a project is likely to be orphan (with justification for the record, like the date of the last official repository's commit, the number of PR/issues that have been opened but never answered…), so that after proper failsafes against abuses, orphan projects can be spotted easily in pip search, on the project's pypi's page and with a warning on pip install.

Transfer maintainership

As a sole maintainer of a few minor projects on pypi, if at some point I disappear from digital life (which I hope will happen in a very very long time ☺) and I'm unable to maintain or transfer ownership of projects I've been working on alone, I don't want to see them becoming orphans, with nobody able to take them over.

I thought maybe a good idea would be to add in the account preferences a person the maintainer trusts blindly, who could act like an "executor of will".

Then if a project is reported as orphan, and someone wants to take it over as a new maintainer to keep it going, administrators of pypa could contact that person to ask for confirmation that the original maintainer is unable to transfer owership himself, and would give permission to transfer ownership on behalf of the original maintainer.

Other processes/ideas?

I just raised the question as I had to deal with a pypi project that's no longer maintained by its author, and get to wonder about how I would want to see my FLOSS projects (which is somehow my "digital legacy") dealt with.

I don't know how similar projects deal with that (e.g. I know that linux distributions like debian has a way to deal with orphan packages, but pypi mostly contains code directly from the author, unlike debian where author ≠ maintainer).
But I have no clue how npm, gem etc. deal with that. if they deal with it at all.

@guyzmo guyzmo changed the title from What to do with abonned/unmaintained projects? to What to do with abandonned/unmaintained projects? Nov 29, 2016

@guyzmo guyzmo changed the title from What to do with abandonned/unmaintained projects? to What to do with abandonned/unmaintained/orphaned projects? Nov 29, 2016

@di

This comment has been minimized.

Member

di commented Nov 29, 2016

We probably also need a real policy/process to release a package name that is being "squatted", i.e. has been registered, but hasn't made any releases. For example, what we did in #1487 seems to have worked, but isn't really a formalized process.

But I have no clue how npm, gem etc. deal with that. if they deal with it at all.

Here's how NPM manages package name disputes: https://docs.npmjs.com/misc/disputes

CPAN has a formal process: http://www.cpan.org/misc/cpan-faq.html#How_adopt_module

RubyGems has an "adoption center" that seems to be defunct: https://groups.google.com/forum/#!msg/rubygems-org/niS5ZO9DNgk/5Fhg9Q3QR7YJ

@FRidh

This comment has been minimized.

FRidh commented Feb 11, 2017

Relevant is the draft of PEP 541
https://www.python.org/dev/peps/pep-0541/

@guyzmo

This comment has been minimized.

guyzmo commented May 20, 2017

BTW, as a related to this topic, I had a simple idea: it's a common usage pattern that a developer looks up the directory to find a library for his project, and finds several that might match the criteria. Maybe could warehouse provide a "score" that would help decision in terms of activity, responsiveness of the project.

When I have to decide, what I currently do is to lookup all projects, then click on the github link, check for the last commit, if there are issues/PRs when were they last attended by the developer(s), and finally choose the one that looks most active. Maybe I could share that feedback within warehouse using a 👍/👎 scheme.

@pradyunsg

This comment has been minimized.

Member

pradyunsg commented May 21, 2017

@guyzmo s/wharehouse/warehouse/ :)

Maybe I could share that feedback within wharehouse using a 👍/👎 scheme.

I believe this has been discussed previously and rejected. IIRC, there won't have voting on Warehouse since it is difficult to moderate the same and at the same time it would disadvantage new packages intended to compete with existing established ones.

could wharehouse provide a "score" that would help decision in terms of activity, responsiveness of the project.

👍 This is worth discussing imo.

Could you check there's is not an issue for something similar and then open a new issue for discussing this idea?

@guyzmo

This comment has been minimized.

guyzmo commented May 28, 2017

Could you check there's is not an issue for something similar and then open a new issue for discussing this idea?

yup there is: cf #991

@guyzmo

This comment has been minimized.

guyzmo commented May 28, 2017

though it's more about a rating, so basically 👍/👎 than really a score that shows how active a project is.

@brainwane

This comment has been minimized.

Member

brainwane commented Mar 20, 2018

I suggest we keep this issue to the issue of abandoned, unmaintained, and orphaned projects, and then cover more general and nuanced project activity statistics in #991.

It looks like PEP 541 is on its way to acceptance. I think we can resolve this issue in the following steps:

  • accept the PEP (this is on the shoulders of the Packaging Working Group)
  • Warehouse developers & Packaging WG decide on logistical stuff (what support queue we point users to for PEP 541 requests, #3231/#1919 dependencies)
  • Warehouse developers coordinate with the Packaging WG to create the extension to the PyPI Terms of Use and link to it in FAQ and PyPI footer, per PEP requirement
  • announce on distutils-sig
@brainwane

This comment has been minimized.

Member

brainwane commented Mar 23, 2018

https://mail.python.org/pipermail/distutils-sig/2018-March/032089.html

PEP 541 accepted! @di shall we move forward on the TODOs above? (Am on mobile)

@di

This comment has been minimized.

Member

di commented Mar 23, 2018

Yep! Let's figure out our process.

I propose we either figure out #3231 or setup pypa/support-requests

Here's the step-by-step process for abandoned projects that I think we should add to our help/policy page somewhere:

  1. Identify the project which is invalid/abandoned
  2. Open a support request at $LOCATION
  3. In the request, demonstrate the criteria for invalid/abandoned projects
  4. In the case of an abandoned project, PyPI admins will start the
    reachability process (6 weeks of attempted contact)
  5. PyPI admins resolve the request
@dstufft

This comment has been minimized.

Member

dstufft commented Mar 23, 2018

One question we'll need to answer is if these should be public or private requests.

@guyzmo

This comment has been minimized.

guyzmo commented Apr 5, 2018

about that, I guess it can be both :

  • use the user's account contact mail first,
  • then submit an issue to the upstream repository (if it has ticket support, and that task can be automated)
@wagner-certat

This comment has been minimized.

wagner-certat commented Apr 23, 2018

What about deceased maintainers? Is it possible to transfer the maintainership in these cases if a death notice is served?

cc @aaronkaplan

@di

This comment has been minimized.

Member

di commented Apr 23, 2018

@wagner-certat Yes, I think that would fall under the category of an "Abandoned Project".

@aaronkaplan

This comment has been minimized.

aaronkaplan commented Apr 23, 2018

@di

This comment has been minimized.

Member

di commented Apr 23, 2018

@wagner-certat We haven't determined the means by which PEP 541 requests will be created yet, see #3231.

Is this a hypothetical issue, or are you attempting to acquire an actual project for which the owner is deceased?

@aaronkaplan

This comment has been minimized.

aaronkaplan commented Apr 23, 2018

@di

This comment has been minimized.

Member

di commented Apr 23, 2018

@aaronkaplan Thanks for the clarification. I'd suggest that you make a separate issue to capture this request. We're currently still defining the process for PEP 541, and once we do we have a large backlog to work through as well, so please have patience.

@aaronkaplan

This comment has been minimized.

aaronkaplan commented Apr 23, 2018

@aaronkaplan

This comment has been minimized.

aaronkaplan commented Apr 23, 2018

okay, separate ticket #3792 created. Please advise on further steps.

@brainwane

This comment has been minimized.

Member

brainwane commented May 6, 2018

(Along the way, someone ought to update the draft "Procedure for Transferring PyPI Name Ownership" Google doc because evidently some people arrive at that still thinking it's the most current process documentation.)

@vidstige

This comment has been minimized.

vidstige commented May 8, 2018

@di @brainwane do we really need the Warehouse support tool (#3231) in order to start processing the backlog of abandoned package claims? Now with PEP-541 finally accepted, the paperwork is in place to process the backlog.

I feel this is currently more of process problem rather than a technical problem: Who will process the support requests? The answer to this question will be no less clear with #3231.

Processing this type of support tickets should take fairly small amount of time per ticket, but will take a lot of calendar time (waiting for eventual answers from current owners). This, in conjunction with the already dire backlog, lends me to believe the community would benefit more from action rather than waiting for more Warehouse features.

@nsheff

This comment has been minimized.

nsheff commented May 8, 2018

Just wanted to add a thought here... have you thought at all about introducing a type of namespace for pypi packages?

With general increase in programming literacy and package development and python's continued popularity growth, package name conflicts are likely to only increase, probably dramatically, over the next few years. Maybe it's worth thinking about employing the method used by github, dockerhub, and others to make the package index use namespaces (so it would be pip install username/packagename). Maybe packages that reach a certain utility (measured by downloads or something) could be granted a spot in the top-level namespace. it seems there are a lot of old, empty, abandoned, or community useless packages consuming high-value top-level names at the moment, and this will get worse.

@anarcat

This comment has been minimized.

anarcat commented May 8, 2018

@nsheff that's a good idea, but in the end that only moves the goalposts around. e.g. on GitHub there are "organizations" that have similar problems than any other toplevel nomenclature. i had to deal with this for the Linkchecker project and the only reason it was easier than on PyPI is that the maintainer on GitHub was different than PyPI: they were responsive and collaborated with the transition.

In other words, while it might, at first look, seem like a nice solution, it won't actually fix the problem for shared namespaces.

Really, at this point I think we just need to figure out who takes care of the support requests. The PEP 541 label in pypi-legacy is probably fine to start with: there are less than 20 projects to deal with right now, and it will probably take longer to discuss a ticketing system and implement it than to just work through those tickets...

For the record, PEP541 says that the "Package Index maintainers" are basically responsible for dealing with such issues, but can escalate to the PSF board which makes recommendations to the Packaging WG who has the final say. But hopefully, most pending issues can just be dispatched easily by the maintainers at this stage. I know that in my case, it's a fairly clear-cut scenario that shouldn't need much more discussion, and I suspect it's probably most of the cases.

I would also suggest adding issue templates to make sure people have gone through all the steps in the process before filing issues, before wasting the maintainers time. ;)

@nsheff

This comment has been minimized.

nsheff commented May 21, 2018

@anarcat you're right that it only moves the goalposts around... but it does move them a lot. if the current, one-namespace method gives you n possible top-level package names, then the github two-level method gives you n^n, which, since n is quite large, is a big difference in space for "good" names... even though it won't solve name conflicts completely, it would dramatically reduce them.

anyway, you're probably right that it's not really necessary at this point... and the other changes will surely improve things, as you say. So, maybe pypi isn't big enough yet for this to make sense. BUT, I'm just wondering if this will become necessary in the future. you can probably find some repository names with dozens or hundreds of duplicates on github (in different user spaces) -- one way to think of it is that github outgrew the 'one-layer' strategy many years ago. pypi packages will never reach the number of github repositories, but two layers is a viable strategy for dealing with name duplication at the scale of github, so that's quite a big goalpost change. maybe that's something to think about as pypi gets more saturated. maybe not this year, but what about next year, or in 5 years? Might be worth thinking about future-proofing things.

@mortenlj

This comment has been minimized.

mortenlj commented May 22, 2018

@nsheff : While I'll agree that your idea has merit, I don't think it is a requirement before deciding on a process and handling the backlog of PEP541 transfers?

I don't see a conflict between deciding on a process and taking care of the PEP541 backlog while discussing/designing a namespace solution for future improvement. I think namespaces for packages could/should be discussed in a separate issue, where it can be the sole focus of the discussion, instead of piggybacking on an almost solved, slightly related issue.

Disclaimer: I'm not involved in pypi development, I'm just waiting for a PEP541 transfer...

@pradyunsg

This comment has been minimized.

Member

pradyunsg commented May 22, 2018

Hey @mortenlj @nsheff @anarcat!

There's #2589 for introducing namespaces for packages. I'd appreciate if you could move the discussion about the it there; since this issue is distinct from that one and likely the best place for PEP 541 updates.

@mfrasca

This comment has been minimized.

mfrasca commented May 25, 2018

The PEP 541 label in pypi-legacy is probably fine to start with: there are less than 20 projects to deal with right now, and it will probably take longer to discuss a ticketing system […]

could it be, that the list is short, because not many people were aware of this possibility?

@nsheff

This comment has been minimized.

nsheff commented Jun 4, 2018

There's #2589 for introducing namespaces for packages. I'd appreciate if you could move the discussion about the it there; since this issue is distinct from that one and likely the best place for PEP 541 updates.

Ah, perfect! sorry to hijack the thread.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment