Policy / procedures for github stale action in the dask org

[gforsyth]

@jacobtomlinson and I met earlier today to talk through some general ideas around how to make better use of labels to prevent issues and PRs getting lost in the shuffle.

There is probot/stale-bot which is a github app run by (surprise) probot. There is also a newer github action that provides (it seems) all of the same functionality.
Since the dask org already makes broad use of github actions, we think that's the way to go:
https://github.com/actions/stale

I just read through the stale action config again and the github action will only apply labels to stale issues if it also comments in the issue, which was discussed during a weekly meeting as something we don't want.
So, probot stale-bot it is, or we can write our own action. Either way:

I propose that we have two "sweeps" of stale checks: short-term and medium-term.

Short-term: The "quick-ish response" pass.
All new issues currently get a "Needs triage" label, this should be extended to new PRs, too.
Any issue or PR that has a needs triage label and hasn't been touched for N days gets a "stale" label, where N is on the order of days. I think 4 days is a reasonable starting point.
For anyone on czar duty, issues/PRs that have a "Needs triage" and a "stale" label are of highest priority to respond to, if only to ping relevant parties or ask for a MRE, etc.

Medium-term: The "gone quiet" pass
Any issue or PR that hasn't been touched in 2(?) weeks gets a stale label added to it.

Exceptions and filtering:
We don't want this to apply to all issues and PRs -- for example, Feature Requests that haven't yet been picked up don't need to be marked as stale. Similarly, issues marked as "good first issue" and "good second issue" have been deliberately left to the wider community, so marking them as stale doesn't make much sense.

The stale-bot seems to only have options for excluding labels (creating an exemption list vs. an inclusion list).

I propose that we exempt issues and PRs with the following labels:

• Good first issue
• good-intro-to-dask
• Good second issue
• Draft or work-in-progress (stale bot doesn't know if a PR is a draft or not)
• Feature request

I'm sure there are more labels that we can/should create and also add to the exemptions.

Finally, doing this will lead very quickly to a bunch of old issues being marked stale.
I think we have a few choices there.

1. We can let that happen, and then gradually triage older "stale" issues, adding labels that will exempt them from future sweeps if we think they should still remain open.
2. We can declare chapter 7 issue bankruptcy for all issues > 2 years old and just close them out
3. We can declare chapter 11 issue bankruptcy for all issues > 2 years old and add a custom label to them to exempt them from stale-bot scans

Thoughts? I'll note that doing this will require more regular label setting from all maintainers, which is a small but consistent extra bit of stuff to have to do.

[martindurant]

Thanks for this write-up.

Noting here as a side-issue, as discussed in the maintenance team: we would like to remove "good first issue", as this has specific meaning for github and encourages python novices to try to claim issues they don't necessarily have the competence for. Instead, we would prefer something like "good-intro-to-dask", i.e., for those that can python already but have not worked specifically on dask before.

[gforsyth]

@mrocklin proposed during the weekly check-in that we handle this with a notion of "active" labels --
all new issues/PRs (and maybe others) get labeled with "Active" and only items with that label are marked as "Needs attention" -- I think we can accomplish that with the following fields:

onlyLabels: ["Active"]
staleLabel: "Needs attention"

[jsignell]

I think all new and all that have been commented on in the past week (or month?) should be labeled active to start with.

[GenevieveBuckley]

I just read through the stale action config again and the github action will only apply labels to stale issues if it also comments in the issue, which was discussed during a weekly meeting as something we don't want.

I don't think this is true, and it looks like the documentation has been updated recently to make that clearer. If you give it an empty string as the message, the bot should not leave a comment. See actions/stale#521

You can skip the comment sending by passing an empty string.

The github action seems like the better option for a few reasons:

1. As explained above, the bot doesn't have to leave comments if we turn that off.
2. You can ignore issues and pull requests before a certain date (so no need to faff with the active/stale workaround being discussed above)
3. As Gil says, Dask already uses github actions for lots of things
4. The project seems a lot more active than the probot one, in terms of maintainer responsiveness

[GenevieveBuckley]

Follow up question, is an "active" or "needs-triage" also required, or was that more being discussed as a workaround to avoid a lot of very old issues being labeled as stale and overwhelming the results (which is not a thing we need to worry about)?

This would be relatively simple to do with a second yaml script using the same action, but I want to double check what the actual workflow is supposed to be. It's not super clear to me what the final decision was here.

[jsignell]

I don't think w a final decision was really arrived at (or if it was it was never implemented), what you are proposing sounds very reasonable to me @GenevieveBuckley

[GenevieveBuckley]

Sorry @jsignell - do you want all new issues/PRs labelled with "needs triage"?

[fjetter]

I would like to revive this discussion. Ironically, it became a bit stale and has been open for almost two years now. There has been a recent change towards how we deal with labels which should be reflected in any of this.

I'm struggling to understand what the outcome of any discussion is, what we decided and what the justification for this decision was.

What I would love to see is

• Everything is automated using a bot
• Issues are flagged as stale after X (30) days
• A comment is placed on stale issues / PRs letting users (and us) know that an issue has been labeled as stale and would like to ask the users to confirm this is still an issue. This will trigger a notification and allows us to act on the stale request, e.g. to label it appropriately because it is something we'd like to keep around
• Issues are automatically closed with a friendly comment after being stale for another 30 days
• All of this is performed for the entire history (see also Stale label github action distributed#5405 (comment))

I understand that some concerns were raised about automatically closing and/or even commenting on tickets. I don't have the full context and would like to understand what led to our rather conservative approach to this so far.

---

For the record, current status of the bot is

• It was merged for dask/dask a couple of months ago Stale label github action dask#8244
• There are currently 44 open tickets labeled as needs attention and only 17 issues closed with this label
• Overall, the dask/dask repo has still 643 open issues so about 7% are labeled with needs attention. Do we think this is accurate? Are the remaining 93% still relevant?
• The dask/distributed PR was never merged Stale label github action distributed#5405
• Both in dask/dask and dask/distributed issues go back to 2015/2016.

[jacobtomlinson]

I am fully supportive of this.