Testing a New Stalebot in GitHub's Community Discussions

[queenofcorgis]

TL;DR - What is happening?

We are working to improve the quality of content across the GitHub Community Discussions. For the next 30 days we are going to perform a test using a custom action to label Question discussions that have not had any activity for 60 days as stale. The action will apply the label stale and comment to inform the original poster of the stale discussion. Sample draft copy is included at the bottom of this announcement.

📢 Feedback is requested on this initial test. Please comment on this post and let us know what you think!

What Discussions will be impacted?

• The action will label discussions that have been labeled Questions and have not had and activity in the past 60 days as stale.
• The action will post the sample copy (included below) as a comment on the discussion to alert the original poster, and request they take action to resolve the question. That automated reply should prompt users to return to add more information, select a reply as the solution, or close the question as out of date. I'll add some example copy we can use below.
• This will only be applied to discussions in the Copilot, Projects and Issues, and Accessibility categories as a part of this initial test.
• This action will not close the discussions automatically.

What is a Stalebot?

You can use GitHub Actions to comment on or close issues that have been inactive for a certain period of time.

Here is more info on this from the GitHub Docs: Closing inactive issues.

We're working with GitHub engineers to test this action in discussions!

Why are only limited categories impacted?

1. We have a hypothesis that using the action to close the loop on open and stale questions in the Community Discussions will improve the quality of content overall, but we want to test assumption before we risk disrupting the entire community discussions experience.
2. The initial test includes categories with high engagement, dedicated community managers, and a high level of involvement from GitHub engineers.
3. We are starting with an initial test with Discussions that are 60 days stale. We want to see how the age of posts impacts overall engagement and resolution rates. We track how long a discussion is open without a response and without an answer so we'll be able to track data on this test and use that to inform our next steps.
4. We want to hear from you! This is an experiment to benefit our users and the quality of content. If it doesn't work, we can roll it back. If it does, we can move forward and apply this across all categories.

What will happen after the trial period?

• The team will have an ongoing evaluation of how the stalebot impacts the response and resolution rates on exiting discussions.
• If we see positive indicators based on data that will be one indicator that this worked
• We will evaluate the feedback on this discussion. If you love it or hate it, we want to know! Please comment with ways you think we can improve the experience.
• If we deploy this broadly, we'll provide users with a hot-to guide here in the Community Discussions so you can apply a similar action to your own repo discussions 🚀

Example Stalebot Copy:

🕒 **Stale Discussion Alert** 🕒

This Discussion has been labeled as stale by an automated system for having no activity in the last 60 days. Please consider one the following actions:

1️⃣ Close as Out of Date: If the topic is no longer relevant, close the Discussion as "out of date" at the bottom of the page.

2️⃣ Provide More Information: Share additional details or context — or let the community know if you've found a solution on your own.

3️⃣ Mark a Reply as Answer: If your question has been answered by a reply, mark the most helpful reply as the solution.

Note: This stale notification will only apply to Discussions with the `Question` label. To learn more, see our recent stalebot announcement.

Thank you for helping bring this Discussion to a resolution! 💬

[ferdnyc]

@queenofcorgis Since you're showing the sample copy, presumably that's meant to invite input, so I have a suggestion:

Note: This stale notification will only apply to Discussions with the Question label. To learn more, see our recent stalebot announcement.

...The message could just link to the "recent stalebot announcement" (that'd be this discussion, #70478), instead of sending us on a scavenger hunt.

[050898056]

b7il2222@gmail.com

[ant-techs]

Hi How can I talk to the creator/Dev of Zello as I have a proposition for him, if you could get in touch at [redacted] I would be grateful to talk with you, Best regards, Anton.

[queenofcorgis]

Hi @ant-techs ,

I've removed your private information from your comment as posting identifiable information is not allowed, per our Code of Conduct.

If you're asking how to use the piece of software that you found in a repository on GitHub, the best way to contact the maintainers of that software is to:

• Check the README for instructions on how to operate the software and pointers to documentation or troubleshooting info.
• Check the SUPPORT file, if one exists, for instructions on how to best contact the maintainers for support.
• Check the CONTRIBUTING guide, if one exists. Sometimes if there isn't a SUPPORT file, the CONTRIBUTING guide will give instructions on how to contact the maintainers for support.

All of these documents can be found, if they exist, in the repository where you found the software itself.

[Marcono1234]

Thanks for being so considerate by only limiting it to the "Question" category and not closing discussions directly.

The stale action (and other similar actions) are known to be quite disruptive in some cases, see actions/stale#719, when they close issues where the maintainer has simply not responded yet. Or in this case here, acting on questions (possibly even well written ones) where no one ever commented on.

As you mentioned, many of the questions here are answered by GitHub staff or at least someone from the community. But some also seem to have no answer at all (discussions with label stale and only a single comment). In that case only action 1, "Close as Out of Date", would be relevant.

Do you think that is a problem, and will you try to address that?

[ant-techs]

[ferdnyc]

@ant-techs

Then you should probably change your GitHub password, because 7 days ago your account posted a message to this discussion that started with, "Hi How can I talk to the creator/Dev of Zello as I have a proposition for him..."

[jsoref]

@queenofcorgis sadly, this is in the announcements category, because I'd really like to see the action mark this discussion as stale.

I'd like to hear if you in fact ran this action and what the results were.

I personally hate the stale bot. (That I send PRs to it is not an endorsement, it's awful software for a lousy purpose.)

There appear to be two constructive pieces of feedback here:

1. If you're running an experiment, you should link to the announcement about the experiment: https://github.com/orgs/community/discussions/70478#discussioncomment-7338056
2. A critique (similar to mine) of the feature: https://github.com/orgs/community/discussions/70478#discussioncomment-7440236

There's also a PR relating to this discussion:
#69778

One problem with bots is that they make a mess of sorting (by last modified) by bubbling up stale items. Sure, it might breathe be life into a thing, but, for a system built around searching, that isn't worth doing. If someone's searching for a thing, they'll find or not find it based on their search and the text of the item.

A better approach would be to improve the UX flow so that authors are encouraged to yes/no answers as they arrive.

[jricciardi]

Hi @jsoref! I can weigh in a little bit here now that we have made some updates and have more data:

With the exception of a few edge case frustrations, we are seeing a lot of overall user acceptance for the stalebot, especially after reducing the "stale" language in favor of "inactive" and "dormant" language.

Additionally, and the numbers change daily, about 5-8% of Discussions are brought to resolution after prompting from the bot; about 1-2% are discovered to actually be product feedback that was incorrectly labeled as a question (this is a clear source of confusion and frustration for some users and something we're paying close attention to); and otherwise most Discussions never get new responses or activity.

A better approach would be to improve the UX flow so that authors are encouraged to yes/no answers as they arrive.

This is great feedback! If we find evidence this type of flow would improve outcomes for the Community, we may escalate this idea.

[jsoref]

I've tripped on the question category thing. The bot didn't explain how to recategorize my feedback.

[jricciardi]

Currently this is something we primarily rely on Community Managers to action on, and don't give wide permissions to avoid malicious users juggling useful Discussions around.

[jsoref]

Well. I have a bunch of items where the bot is complaining. They were generally meant as product feedback, not "questions".

I just walked home from work because I'm still recovering from a concussion, so if a community manager could please fix mine, I'd appreciate it. I can't find the one that was just touched today, and my headache is getting worse.

[ferdnyc]

@jsoref

One problem with bots is that they make a mess of sorting (by last modified) by bubbling up stale items.

That's true, though the stalebot does also label the discussions it wades into, helpfully. That means there's always the possibility of adding -label:inactive to the search/filter terms, in order to exclude those items.

It's not ideal, but it can at least help mitigate some of the undesirable (or at least, debatably-desirable) effects.

[timmc-edx]

A bit of feedback: I'm here because the stalebot just replied to a question I've had open for some months about the GitHub API. It suggests three courses of action, all of which are frustrating to read:

• "Close as Out of Date": Nope, I still have the problem.
• "Provide More Information": Already gave all the info I have.
• "Mark a Reply as Answer": No one has answered, so there's no reply to mark.

The only useful action I could take would be something like "Do $X to get a GitHub employee to look at your question", but that's not one of the suggestions. :-) So the bot is just reminding me that there's a gap in GH's API that I can't do anything about, and there's no one to contact, and no one has responded... which doesn't feel great.

Anyway, I don't have a specific thing I think you should do differently (with regards to the bot at least) but I did want to share that the net effect was very annoying.

[jsoref]

I see a Product Feedback tag which seems reasonable.

[timmc-edx]

Hmm! It wasn't there when I asked. Maybe it got manually labeled? Anyway, I guess the process works somehow. :-)

[jsoref]

https://github.com/community/community/blob/main/.github/workflows/label-templated-discussions.yml
https://github.com/community/community/actions/runs/9101979677/job/25020504711#step:2:1

[jricciardi]

@timmc-edx Yep, that's applied by the Action. We love a good Action here :)

And thank you for submitting that as product feedback! 🚀

[jsoref]

This is going to happen to me a lot. I don't think that "Question" vs "Product Feedback" made such a distinction to me when I filed things, so I bet a fairly large number of my items will get this noise and result in me refiling things.

I viewed "Question" as "why is this product doing X?" or "could this product do Y better?". I imagine I'd use "Product Feedback" for "Z is broken".

I have a headache and an not going to try to do a systematic review of all of my discussion posts. -- it's possible that the method I used to create them mattered.

Today I'm using a mobile browser (where the discussion category was not as visible and easy enough to skip past), whereas for filing issues about GitHub Mobile, I tend to use GitHub Mobile, and otherwise I'll generally be using a desktop browser.

https://github.com/orgs/community/discussions/81741

[BoogieAlastor123]

[jsoref]

https://github.com/orgs/community/discussions?discussions_q=is%3Aopen+label%3AQuestion+%22Discussion+Activity+Reminder%22

[MasterInQuestion]

    https://github.com/orgs/community/discussions/39375#discussioncomment-9588019
    https://github.com/orgs/community/discussions/108088#discussioncomment-9588003

    Not contributive in some cases.

[ferdnyc]

As I commented there, your second link actually turned out to be an argument for the stalebot. Its poking of that discussion ultimately led to additional comments being posted and, from those, an answer being selected.

[MasterInQuestion]

[[
[ MasterInQuestion @ CE 2024-05-29 14:44:51 UTC:
https://github.com/orgs/community/discussions/108088#discussioncomment-9594805
    Maybe mere notifying the author shall be more appropriate, rather than posting here:
    Creating pointless noise mostly irrelevant. ]

----

[ ferdnyc @ CE 2024-05-29 22:41:47 UTC:
https://github.com/orgs/community/discussions/108088#discussioncomment-9598960
    That's a very strange argument to make:
    When the result of the stalebot's posting was, ultimately:
    That someone other than the author, posted comments:
    That ultimately resulted in an answer being selected.

    I'm not saying the stalebot made that happen.
    But it calling attention to the discussion clearly had the desired effect: notifying the author alone would not have. ]

----

[ ferdnyc @ CE 2024-05-29 22:44:31 UTC:
https://github.com/orgs/community/discussions/70478#discussioncomment-9598985
    As I commented there, your 2nd link actually turned out to be an argument for the stalebot:
    Its poking of that discussion ultimately led to additional comments being posted.
    .
    And, from those: an answer being selected. ]
]]

    And this is the selected answer..?
    https://github.com/orgs/community/discussions/108088#discussioncomment-9595059

    Whose another reply there doesn't seem to be of much use:
    Mere repetition, and somewhat misleading. (non-equivalent: how the same?)

    Who has also, in addition: spammed bunch circular inexplicable links of duplicate elsewhere.
    (e.g. right in this thread [ https://github.com/orgs/community/discussions/70478#discussioncomment-9609043 ])


    As demonstrated:
    The bot message is completely off-topic and adds no value in the long term:
    Only pointlessly polluting the thread. (and potentially mistag)

    If the instigation is what really wanted: same can be achieved without piling the verbiage.
    Less verbosity = More readability

[jsoref]

@ferdnyc: I agree with @MasterInQuestion, this wasted a considerable amount of my time as well as the questioners and anyone subscribed.

I'm only adding these admittedly fairly useless comments because the bot is asking me to do so. -- In the case of the selected answer, it wasn't technically useless as it pointed to how the Questioner themselves solved the problem (and when I suggested marking something as the solution, I really meant the Questioner's post saying they'd solved it, more than my own response) -- It's true that the other post was more valuable to the next person as it provided an explanation as to why that change was correct, but...

Worse, each of my interactions risks someone accusing me of misbehaving. Whether that's link spam, abuse, gaming, ...

I don't need/want to take any of these actions. But, I'm invested in getting my tickets addressed. -- And they won't be addressed if they're closed by a bot.

It's true that some tickets may make slight progress. But there are/have-to-be better approaches to achieve that aim.

Contributors who positively improve tickets (by either answering questions, or marking questions as answered, for which answers aren't later rejected by the original poster) could be given action-credits or something else of value within the GitHub ecosystem (I certainly could use those credits). Obviously if one went this way, abuse would have to result in people not being able to receive such rewards, but that's also fairly easy to administer.

[MasterInQuestion]

[[
    I'm only adding these admittedly fairly useless comments because the bot is asking me to do so.

    In the case of the selected answer, it wasn't technically useless: as it pointed to how the questioner themselves solved the problem.
    (and when I suggested marking something as the solution: I really meant the questioner's post saying they'd solved it, more than my own response)
    .
    It's true that the other post was more valuable to the next person:
    As it provided an explanation as to why that change was correct, but...

    Worse, each of my interactions risks someone accusing me of misbehaving.
    Whether that's link spam, abuse, gaming, ...

    I don't need/want to take any of these actions.
    But, I'm invested in getting my tickets addressed.
    And they won't be addressed if they're closed by a bot.

    It's true that some tickets may make slight progress.
    But there are (or must be) better approaches to achieve that aim.

    Contributors who positively improve tickets:
    |*| By either answering questions
    |*| Or marking questions as answered
    |*| For which answers aren't later rejected by the original poster
    ; could be given action-credits or something else of value within the GitHub ecosystem.
    (I certainly could use those credits)
    .
    Obviously if one went this way: abuse would have to result in people not being able to receive such rewards.
    But that's also fairly easy to administer.
]]
    I didn't say the selected answer is useless: merely have some doubt on it.
    (not strong enough to present an objection)

    Regardless, feel free to practice what you would like: as you feel happy about it.

    Note:
    Closed issues/discussions/pulls whatsoever doesn't mean no more accepting future actions. (i.e. locked)
    People may still freely continue on which.
    .
    The Closed/Open status mostly merely serves as some sort of tag. (for mere identification)

[adamlui]

I raised a bug https://github.com/orgs/community/discussions/88005 and Question is just auto-applied and now it's marked dormant. I assume it will be swiftly closed. This does not help in any way, instead just wastes the time of those who make bug reports. An employee needs to address/fix it instead of let bots bury them

[jricciardi]

Hey there @adamlui - It looks like this Discussion was originally posted as a Question and then you edited to be a Bug, but it still had the Question label on it, triggering the bot. That's an edge case we'll have to think through solutions for. Thank you for flagging!

I've gone ahead and relabeled that Discussion for you and removed the inactive label 👍

[adamlui]

@jricciardi every bug I've ever raised got Question-labeled and user cannot remove labels iirc

[jsoref]

@queenofcorgis / @jricciardi: I've refiled a couple as a courtesy to your team (https://github.com/orgs/community/discussions/126580, https://github.com/orgs/community/discussions/126631, https://github.com/orgs/community/discussions/126632).

I'd appreciate it if you were to edit the labels for these remaining three from Question to Product Feedback:

https://github.com/orgs/community/discussions/60448
https://github.com/orgs/community/discussions/58143
https://github.com/orgs/community/discussions/39447

Sincerely,
Josh Soref (@jsoref 🤕 not recovering well from a concussion, and not appreciating unpaid busy work)