New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consolidate version-specific untriaged queries. #43

Open
dolske opened this Issue Jun 5, 2018 · 3 comments

Comments

Projects
None yet
2 participants
@dolske
Copy link

dolske commented Jun 5, 2018

Currently AWTY shows results broken down by version. e.g. for a specific version or all versions (where the all-versions page just shows the nightly/beta/release reports concatenated together.)

I think this breakdown ends up not working well -- many of the bugs are repeated across lists, it makes it difficult to get an overview of the state of overall triage, and it makes triagers work more difficult.

The current queries break out "versions" by using the date of when the bug was filed. e.g. Firefox 60 started on m-c on Jan 22, so the query filters for any bug filtered after that date. Firefox 62 started on m-c on May 7th, so the Fx62 query similarly filters. But that means the Fx60 query includes all Fx62 bugs. This alone is not technically wrong (since most bugs lack more detailed metadata about exactly what versions they affect), but it means there can be a lot of duplication across "versions".

For example, at the moment AWTY says that for Firefox::Untriaged, there are 136 untriaged bugs for Fx60, and 127 for Fx62. (The actual numbers, upon clicking the number, are 127 for Fx60 and 117 for Fx62. Caching I assume.) But the vast majority of those bugs are the same -- diffing the list, there are just 10 bugs different between those two queries. If you look at those Fx60 bugs, all but 10 were filed after May 7th. This also means you can't just add up the untriaged count, by version, to get a total number of untriaged bugs, because you'd be double-counting.

But since the queries for 60/61/62 are all slightly different (because of the status-firefoxXX flag checks) this is a pain for triaging (as I've noted before) because the lists can be slightly different, and thus triage owners need to go through each component they own 3 times. (Firefox::Untriaged is a bad example for this case, because status-firefoxXX flags are rarely going to be set in this component.) Triagers should, ideally, have a single link and a single number to monitor.

IMO, the simplest thing is to just collapse this all down to a single query that doesn't try to show status by version. It can still use the bug-filed-after cutoff date of the current Release, and should filter on "(status-firefox60 == ?/---/affected OR status-firefox61 == ?/---/affected OR status-firefox62 == ?/---/affected)".

In other words, a bug filed since date X that is/may be affecting a current release, and will show up on the list unless it's explicitly marked as (say) "wontfix" for all of nightly/beta/release. [There might even be an argument for removing this entirely – for triagers – since the goal is to to be assigning a priority to bugs within a week or two of it being filed. At that point, it's not terribly useful to get bugs off the radar by kicking the can down the road with status-firefoxXX flags.]

I suppose release-drivers will still want something more limited by version (and that's definitely checking status-firefoxXX flags) to monitor bugs that may affect something heading out the door, but that feels like a different use-case than basic triage.

@emceeaich

This comment has been minimized.

Copy link
Owner

emceeaich commented Jun 8, 2018

I talked about this with @past and think something like the queries we use for the weekly regression triage could be filters on that one query. Three of the filters would be: affecting release, affecting beta, affecting nightly. A bug that affects release, beta, and nightly, or affecting release and beta would be in the first list, and so on. The fourth and most likely largest list would be the bugs for which we don't know if a list is affected.

The change would be to query over the whole set of bugs since the current release branched, ignoring the ones which we know we're not addressing in any train, the:

"(status-firefox60 == ?/---/affected OR status-firefox61 == ?/---/affected OR status-firefox62 == ?/---/affected)"

from above. And then applying those filters from above. The other lists (P1s affecting or may affect, fix or defer, and uplifted) could move to another dashboard.

@emceeaich

This comment has been minimized.

Copy link
Owner

emceeaich commented Aug 14, 2018

Just an update, I need to schedule a "work week" on dashboards so I can get this in along with some other changes I want to make.

@emceeaich emceeaich self-assigned this Nov 19, 2018

@emceeaich

This comment has been minimized.

Copy link
Owner

emceeaich commented Nov 20, 2018

@dolske, here's what that looks like for just the untriaged bugs.

Now with corrected totals.

screenshot_2018-11-20 are we triaged yet 1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment