Updates to our triage guidelines #4180

agjohnson · 2018-06-01T00:20:13Z

A few things I've wanted as I do PM, plus some changes for later automation via
probot:

Drop Enhancement -- it vaguely implies "new feature"
Instead use Feature for new accepted and on our roadmap features
If an issue doesn't describe a bug, and is not a new feature, it's likely an
Improvement. The distinction here is Feature is a new feature, Bug
has higher priority than Improvement
Needed: more information is a way to track issues that require a reply and
can be closed if we don't get that reply. Currently, we use this in some
places as a way to describe issues that we need to collect more ideas on --
a good substitute is Needed: design decision
Also add Status: stale for issue automation
Drop unused statuses for closed issues
Drop PR: ready as this is a leftover from before GH had reviews
Drop Feaure Overview, this is synonymous with Feature

The takeaway here for purposes of what to audit with these changes in place:

If an issue is an Enhancement now:
- It becomes a Feature if it is a new feature and belongs on our roadmap
- It becomes an Improvement if it is not a bug and belongs on our roadmap
- It gets closed otherwise, it's not important to us
Issues that are Needed: more information can be closed if there was no reply
after 7 days. We might need to retriage some issues where this label was
applied for another reason

Also, suggestions for alternatives to Improvement are helpful, if this isn't
descriptive enough.

A few things I've wanted as I do PM, plus some changes for later automation via probot: * Drop `Enhancement` -- it vaguely implies "new feature" * Instead use `Feature` for new **accepted** and on our roadmap features * If an issue doesn't describe a bug, and is not a new feature, it's likely an Improvement. The distinction here is `Feature` is a new feature, `Bug` has higher priority than `Improvement` * `Needed: more information` is a way to track issues that require a reply and can be closed if we don't get that reply. Currently, we use this in some places as a way to describe issues that we need to collect more ideas on -- a good substitute is `Needed: design decision` * Also add `Status: stale` for issue automation * Drop unused statuses for closed issues * Drop PR: ready as this is a leftover from before GH had reviews * Drop `Feaure Overview`, this is synonymous with `Feature` The takeaway here for purposes of what to audit with these changes in place: * If an issue is an `Enhancement` now: * It becomes a `Feature` if it is a new feature and belongs on our roadmap * It becomes an `Improvement` if it is not a bug and belongs on our roadmap * It gets closed otherwise, it's not important to us * Issues that are `Needed: more information` can be closed if there was no reply after 7 days. We might need to retriage some issues where this label was applied for another reason Also, suggestions for alternatives to `Improvement` are helpful, if this isn't descriptive enough.

humitos · 2018-06-04T21:46:03Z

I wouldn't replace Needed: more information with Needed: design decision. The first one means that we are waiting for more info to help us debug or to understand what is being requested (usually, from the user. Although, it could be from ourselves also). The later is for the core team to make a decision on how to implement something or if implement a requested feature or not, for example.

With and strict "belongs on our roadmap" we will end up with many issues without tags at all because the person that is triagging may not be sure if we want to implement it or not. I suppose that those can be marked as Needed: design decision but we will have a lot of these. I think we should have some way to mark issues as "Enhancement" or so where the triagging person is in doubt if that will be implement in the end. We can achieve this by marking as Needed: design decision and Feature/Improvement maybe.

humitos · 2018-06-04T21:48:38Z

docs/issue-labels.rst

-    process <triage-not-enough-information>` for more information.
+    This label indicates that a reply with more information is required from the
+    bug reporter. If no response is given by the reporter, the issue is
+    considered invalid after 7 days and will be closed. See the documentation


7 days is not enough time, I'd say. At least, we will want 2 weeks.

agjohnson · 2018-06-04T23:31:50Z

I wouldn't replace Needed: more information with Needed: design decision.

The idea isn't to make this replacement. It's that more information is a label reserved for interacting with non-core users -- a way to track issues that can be closed when we don't get what we need to act on the issue.

Arguably, core should not be creating issues that would end up in a more information state. If a core dev can't provide enough information for the ticket, it probably shouldn't be a ticket. If this is to replicate a bug Needed: replication is a status that implies we still need to investigate the bug. If the core dev is trying to create a new feature and wants to communicate they are still researching a solution/etc, design decision communicates this, as well as that other core team should be able to drop in and provide information or help.

With and strict "belongs on our roadmap" we will end up with many issues without tags at all because the person that is triagging may not be sure if we want to implement it or not. I suppose that those can be marked as Needed: design decision

Yup, that would be the idea. If the issue is a new Feature, we don't want to blindly accept the new feature to our roadmap. This should either be dropped or go through a design decision phase. From there, it either becomes an accepted feature, or it is closed.

I don't get much of any value of having issues labeled Enhancement right now. It's basically just a catch-all label for "this issue is not support or a bug".

ericholscher · 2018-06-05T15:12:49Z

+1 for:

more information is for info from users
design decision is for info from maintainers

+1 on having a more explicit "Accepted" state, which defines work that we agree should be done, instead of just something that exists in the bug tracker.

ericholscher

This looks like a good improvement. This document is specifying a lot of process that we don't have support for. Are we scheming putting a bot behind this to implement the procedures?

We also don't have a real procedure in place to make sure that Bugs & Accepted Features get priority. That's work we need to do, but we should make sure to follow through on the implementation of it.

I'm also a little hesitant to auto-close all things after a set amount of time. I'd be more willing to have that be scoped to a specific set of tags or issues (eg. Improvements but not Accepted Features or Bugs)

ericholscher · 2018-06-05T15:14:49Z

docs/issue-labels.rst

    worth to fix or implement in the future. However the core team's resources
-    are too scarce to address these issues. Tickets marked with this label
+    are too scarce to address these issues. Issues marked with this label
    are issues that the core team will **not** work on, but contributions
    from the community are very welcome.


I'd be tempted to just close these and remove the concept, as we've talked about in the past (#2673)

Good point. These generally just sit around without activity. I think just closing these as non-accepted features and invalid support issues is in line with the rest of the changes here.

ericholscher · 2018-06-05T15:15:26Z

docs/issue-labels.rst

+    issues that may become features some day. Issues that do not describe
+    new features, such as code cleanup or fixes that are not related to a bug,
+    should probably be given the Improvement label instead. On release, issues
+    with the Feature label warrant at least a minor version increase.


I'd be more explicit here, and call it an "Accepted Feature", to make the intent more clear.

I noted before this, maybe an Accepted tag would be another addition. Does that add too much variability to issue state? We'd have:

Accepted + Feature -- makes sense

Accepted + Improvement -- also makes sense

Accepted + Bug -- i guess this makes sense too, we do have some bugs in a non-accepted sort of state. They are off our roadmap and/or not replicated.

Yea, I think Accepted Bug could also be verified/triaged, as well as "something we want to fix".

ericholscher · 2018-06-05T15:17:13Z

docs/issue-labels.rst

+*Status: stale*
+    A issue is stale if it there has been no activity on it for 90 days. Once a
+    issue is determined to be stale, it will be closed in 7 days unless there
+    is activity on the issue.


Not sure the real value here. Do we imagine a bot adding a stale tag, and then that would magically get someone to update the ticket?

Yeah, this is strictly for probot automation in #4111

ericholscher · 2018-06-05T15:20:07Z

docs/issue-labels.rst

+    An issue describing unexpected or malicious behaviour of the readthedocs.org
+    software. A Bug issue differs from an Improvement issue in that Bug issues
+    are given priority on our roadmap. On release, these issues generally only
+    warrant incrementing the patch level version.


Do we need a distinction between Bugs & Accepted Bugs, similar to Features/Improvements? It feels like we'd want to have some kind of two-step there as well.

I'd be +1 on this

ericholscher · 2018-06-05T15:20:56Z

cc @RichardLitt -- I imagine you have some feedback on this as well.

agjohnson · 2018-06-07T15:49:40Z

+1 on having a more explicit "Accepted" state, which defines work that we agree should be done, instead of just something that exists in the bug tracker.

I like this -- perhaps we just need an Accepted label, to be really explicit. I imply issues are accepted by putting them on a versioned milestone, but an explicit label would be more helpful for non-core i'm sure.

Are we scheming putting a bot behind this to implement the procedures?

Yeah, some of this is preparing for some automation which isn't set up yet.

I'm also a little hesitant to auto-close all things after a set amount of time.

Yeah same. I think in the case of probot automation, we'd target just Support issues for now. I worry about losing historical issues and issues that we've just had to let go stale.

ericholscher · 2018-06-07T16:11:57Z

+1 on Accepted as a label, and then keeping the additional classifications.

RichardLitt · 2018-06-07T20:41:01Z

@ericholscher I saw this, and gave it a 👍 :D. I think it all sounds great, and I had no substantive comments to make. @agjohnson, you wrote a good issue here.

Priority: high already exists in rtfd/readthedocs.org. Matching low priority for extra use as well.

agjohnson · 2018-06-08T18:27:10Z

Updated PR the feedback, also added some existing changes we already have in issues.

Addresses readthedocs/readthedocs.org#4180 Also is the foundation for better autochangelog

humitos

I'm 👍 on this.

I think with this pattern the triagge person can label the issues with proper labels like Bug or Feature, etc but there is a final step required from the core team to add the Accepted which I think it's pretty good.

agjohnson requested a review from a team June 1, 2018 00:20

agjohnson added this to the Development improvements milestone Jun 1, 2018

agjohnson added the Needed: design decision A core team decision is required label Jun 1, 2018

humitos reviewed Jun 4, 2018

View reviewed changes

humitos mentioned this pull request Jun 4, 2018

Add Probots #4111

Closed

ericholscher reviewed Jun 5, 2018

View reviewed changes

agjohnson added 3 commits June 8, 2018 10:37

7d -> 2w, drop Community Effort, add Accepted label

d981892

Also make Priority: high and Priority: low

595440e

Priority: high already exists in rtfd/readthedocs.org. Matching low priority for extra use as well.

Add PR hotfix

0102682

agjohnson added a commit to readthedocs/common that referenced this pull request Jun 8, 2018

Automate repo label syncing

484f84a

Addresses readthedocs/readthedocs.org#4180 Also is the foundation for better autochangelog

agjohnson mentioned this pull request Jun 8, 2018

Automate repo label syncing readthedocs/common#9

Merged

humitos approved these changes Jun 8, 2018

View reviewed changes

agjohnson merged commit f9316b9 into master Jun 8, 2018

agjohnson deleted the agj/triage-update branch June 8, 2018 21:53

This was referenced Jun 18, 2018

Drop Community Effort issue designation #2673

Closed

Update docs to match the new triague guidelines #4260

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Updates to our triage guidelines #4180

Updates to our triage guidelines #4180

agjohnson commented Jun 1, 2018

humitos commented Jun 4, 2018

humitos Jun 4, 2018

agjohnson commented Jun 4, 2018

ericholscher commented Jun 5, 2018

ericholscher left a comment •

edited

Loading

ericholscher Jun 5, 2018

agjohnson Jun 7, 2018

ericholscher Jun 5, 2018

agjohnson Jun 7, 2018

ericholscher Jun 7, 2018

ericholscher Jun 5, 2018

agjohnson Jun 7, 2018

ericholscher Jun 5, 2018

agjohnson Jun 7, 2018

ericholscher commented Jun 5, 2018 •

edited

Loading

agjohnson commented Jun 7, 2018

ericholscher commented Jun 7, 2018

RichardLitt commented Jun 7, 2018

agjohnson commented Jun 8, 2018

humitos left a comment

Updates to our triage guidelines #4180

Updates to our triage guidelines #4180

Conversation

agjohnson commented Jun 1, 2018

humitos commented Jun 4, 2018

Choose a reason for hiding this comment

agjohnson commented Jun 4, 2018

ericholscher commented Jun 5, 2018

ericholscher left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ericholscher commented Jun 5, 2018 • edited Loading

agjohnson commented Jun 7, 2018

ericholscher commented Jun 7, 2018

RichardLitt commented Jun 7, 2018

agjohnson commented Jun 8, 2018

humitos left a comment

Choose a reason for hiding this comment

ericholscher left a comment •

edited

Loading

ericholscher commented Jun 5, 2018 •

edited

Loading