Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updates to our triage guidelines #4180

Merged
merged 4 commits into from Jun 8, 2018
Merged

Updates to our triage guidelines #4180

merged 4 commits into from Jun 8, 2018

Conversation

@agjohnson
Copy link
Contributor

@agjohnson agjohnson commented Jun 1, 2018

A few things I've wanted as I do PM, plus some changes for later automation via
probot:

  • Drop Enhancement -- it vaguely implies "new feature"
  • Instead use Feature for new accepted and on our roadmap features
  • If an issue doesn't describe a bug, and is not a new feature, it's likely an
    Improvement. The distinction here is Feature is a new feature, Bug
    has higher priority than Improvement
  • Needed: more information is a way to track issues that require a reply and
    can be closed if we don't get that reply. Currently, we use this in some
    places as a way to describe issues that we need to collect more ideas on --
    a good substitute is Needed: design decision
  • Also add Status: stale for issue automation
  • Drop unused statuses for closed issues
  • Drop PR: ready as this is a leftover from before GH had reviews
  • Drop Feaure Overview, this is synonymous with Feature

The takeaway here for purposes of what to audit with these changes in place:

  • If an issue is an Enhancement now:
    • It becomes a Feature if it is a new feature and belongs on our roadmap
    • It becomes an Improvement if it is not a bug and belongs on our roadmap
    • It gets closed otherwise, it's not important to us
  • Issues that are Needed: more information can be closed if there was no reply
    after 7 days. We might need to retriage some issues where this label was
    applied for another reason

Also, suggestions for alternatives to Improvement are helpful, if this isn't
descriptive enough.

A few things I've wanted as I do PM, plus some changes for later automation via
probot:

* Drop `Enhancement` -- it vaguely implies "new feature"
* Instead use `Feature` for new **accepted** and on our roadmap features
* If an issue doesn't describe a bug, and is not a new feature, it's likely an
  Improvement. The distinction here is `Feature` is a new feature, `Bug`
  has higher priority than `Improvement`
* `Needed: more information` is a way to track issues that require a reply and
  can be closed if we don't get that reply. Currently, we use this in some
  places as a way to describe issues that we need to collect more ideas on --
  a good substitute is `Needed: design decision`
* Also add `Status: stale` for issue automation
* Drop unused statuses for closed issues
* Drop PR: ready as this is a leftover from before GH had reviews
* Drop `Feaure Overview`, this is synonymous with `Feature`

The takeaway here for purposes of what to audit with these changes in place:

* If an issue is an `Enhancement` now:
  * It becomes a `Feature` if it is a new feature and belongs on our roadmap
  * It becomes an `Improvement` if it is not a bug and belongs on our roadmap
  * It gets closed otherwise, it's not important to us
* Issues that are `Needed: more information` can be closed if there was no reply
  after 7 days. We might need to retriage some issues where this label was
  applied for another reason

Also, suggestions for alternatives to `Improvement` are helpful, if this isn't
descriptive enough.
@humitos
Copy link
Member

@humitos humitos commented Jun 4, 2018

I wouldn't replace Needed: more information with Needed: design decision. The first one means that we are waiting for more info to help us debug or to understand what is being requested (usually, from the user. Although, it could be from ourselves also). The later is for the core team to make a decision on how to implement something or if implement a requested feature or not, for example.

With and strict "belongs on our roadmap" we will end up with many issues without tags at all because the person that is triagging may not be sure if we want to implement it or not. I suppose that those can be marked as Needed: design decision but we will have a lot of these. I think we should have some way to mark issues as "Enhancement" or so where the triagging person is in doubt if that will be implement in the end. We can achieve this by marking as Needed: design decision and Feature/Improvement maybe.

process <triage-not-enough-information>` for more information.
This label indicates that a reply with more information is required from the
bug reporter. If no response is given by the reporter, the issue is
considered invalid after 7 days and will be closed. See the documentation
Copy link
Member

@humitos humitos Jun 4, 2018

7 days is not enough time, I'd say. At least, we will want 2 weeks.

@humitos humitos mentioned this pull request Jun 4, 2018
@agjohnson
Copy link
Contributor Author

@agjohnson agjohnson commented Jun 4, 2018

I wouldn't replace Needed: more information with Needed: design decision.

The idea isn't to make this replacement. It's that more information is a label reserved for interacting with non-core users -- a way to track issues that can be closed when we don't get what we need to act on the issue.

Arguably, core should not be creating issues that would end up in a more information state. If a core dev can't provide enough information for the ticket, it probably shouldn't be a ticket. If this is to replicate a bug Needed: replication is a status that implies we still need to investigate the bug. If the core dev is trying to create a new feature and wants to communicate they are still researching a solution/etc, design decision communicates this, as well as that other core team should be able to drop in and provide information or help.

With and strict "belongs on our roadmap" we will end up with many issues without tags at all because the person that is triagging may not be sure if we want to implement it or not. I suppose that those can be marked as Needed: design decision

Yup, that would be the idea. If the issue is a new Feature, we don't want to blindly accept the new feature to our roadmap. This should either be dropped or go through a design decision phase. From there, it either becomes an accepted feature, or it is closed.

I don't get much of any value of having issues labeled Enhancement right now. It's basically just a catch-all label for "this issue is not support or a bug".

@ericholscher
Copy link
Member

@ericholscher ericholscher commented Jun 5, 2018

+1 for:

  • more information is for info from users
  • design decision is for info from maintainers

+1 on having a more explicit "Accepted" state, which defines work that we agree should be done, instead of just something that exists in the bug tracker.

Copy link
Member

@ericholscher ericholscher left a comment

This looks like a good improvement. This document is specifying a lot of process that we don't have support for. Are we scheming putting a bot behind this to implement the procedures?

We also don't have a real procedure in place to make sure that Bugs & Accepted Features get priority. That's work we need to do, but we should make sure to follow through on the implementation of it.

I'm also a little hesitant to auto-close all things after a set amount of time. I'd be more willing to have that be scoped to a specific set of tags or issues (eg. Improvements but not Accepted Features or Bugs)

worth to fix or implement in the future. However the core team's resources
are too scarce to address these issues. Tickets marked with this label
are too scarce to address these issues. Issues marked with this label
are issues that the core team will **not** work on, but contributions
from the community are very welcome.
Copy link
Member

@ericholscher ericholscher Jun 5, 2018

I'd be tempted to just close these and remove the concept, as we've talked about in the past (#2673)

Copy link
Contributor Author

@agjohnson agjohnson Jun 7, 2018

Good point. These generally just sit around without activity. I think just closing these as non-accepted features and invalid support issues is in line with the rest of the changes here.

issues that may become features some day. Issues that do not describe
new features, such as code cleanup or fixes that are not related to a bug,
should probably be given the Improvement label instead. On release, issues
with the Feature label warrant at least a minor version increase.
Copy link
Member

@ericholscher ericholscher Jun 5, 2018

I'd be more explicit here, and call it an "Accepted Feature", to make the intent more clear.

Copy link
Contributor Author

@agjohnson agjohnson Jun 7, 2018

I noted before this, maybe an Accepted tag would be another addition. Does that add too much variability to issue state? We'd have:

  • Accepted + Feature -- makes sense
  • Accepted + Improvement -- also makes sense
  • Accepted + Bug -- i guess this makes sense too, we do have some bugs in a non-accepted sort of state. They are off our roadmap and/or not replicated.

Copy link
Member

@ericholscher ericholscher Jun 7, 2018

Yea, I think Accepted Bug could also be verified/triaged, as well as "something we want to fix".

*Status: stale*
A issue is stale if it there has been no activity on it for 90 days. Once a
issue is determined to be stale, it will be closed in 7 days unless there
is activity on the issue.
Copy link
Member

@ericholscher ericholscher Jun 5, 2018

Not sure the real value here. Do we imagine a bot adding a stale tag, and then that would magically get someone to update the ticket?

Copy link
Contributor Author

@agjohnson agjohnson Jun 7, 2018

Yeah, this is strictly for probot automation in #4111

An issue describing unexpected or malicious behaviour of the readthedocs.org
software. A Bug issue differs from an Improvement issue in that Bug issues
are given priority on our roadmap. On release, these issues generally only
warrant incrementing the patch level version.
Copy link
Member

@ericholscher ericholscher Jun 5, 2018

Do we need a distinction between Bugs & Accepted Bugs, similar to Features/Improvements? It feels like we'd want to have some kind of two-step there as well.

Copy link
Contributor Author

@agjohnson agjohnson Jun 7, 2018

I'd be +1 on this

@ericholscher
Copy link
Member

@ericholscher ericholscher commented Jun 5, 2018

cc @RichardLitt -- I imagine you have some feedback on this as well.

@agjohnson
Copy link
Contributor Author

@agjohnson agjohnson commented Jun 7, 2018

+1 on having a more explicit "Accepted" state, which defines work that we agree should be done, instead of just something that exists in the bug tracker.

I like this -- perhaps we just need an Accepted label, to be really explicit. I imply issues are accepted by putting them on a versioned milestone, but an explicit label would be more helpful for non-core i'm sure.

Are we scheming putting a bot behind this to implement the procedures?

Yeah, some of this is preparing for some automation which isn't set up yet.

I'm also a little hesitant to auto-close all things after a set amount of time.

Yeah same. I think in the case of probot automation, we'd target just Support issues for now. I worry about losing historical issues and issues that we've just had to let go stale.

@ericholscher
Copy link
Member

@ericholscher ericholscher commented Jun 7, 2018

+1 on Accepted as a label, and then keeping the additional classifications.

@RichardLitt
Copy link
Member

@RichardLitt RichardLitt commented Jun 7, 2018

@ericholscher I saw this, and gave it a 👍 :D. I think it all sounds great, and I had no substantive comments to make. @agjohnson, you wrote a good issue here.

agjohnson added 3 commits Jun 8, 2018
Priority: high already exists in rtfd/readthedocs.org. Matching low priority for
extra use as well.
@agjohnson
Copy link
Contributor Author

@agjohnson agjohnson commented Jun 8, 2018

Updated PR the feedback, also added some existing changes we already have in issues.

agjohnson added a commit to readthedocs/common that referenced this issue Jun 8, 2018
Addresses readthedocs/readthedocs.org#4180

Also is the foundation for better autochangelog
humitos
humitos approved these changes Jun 8, 2018
Copy link
Member

@humitos humitos left a comment

I'm 👍 on this.

I think with this pattern the triagge person can label the issues with proper labels like Bug or Feature, etc but there is a final step required from the core team to add the Accepted which I think it's pretty good.

@agjohnson agjohnson merged commit f9316b9 into master Jun 8, 2018
1 check passed
@agjohnson agjohnson deleted the agj/triage-update branch Jun 8, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

4 participants