dashboard/app: auto-close linux-next bugs with repros

[dvyukov]

Normally for bugs with repros we can do fix bisection and that's the plan for closure.
But we can't do fix bisection for linux-next bugs, so it can make sense to retest on head and auto-close if it does not happen anymore.
Should we do this for linux-next-only bugs? We may have a bug that happens on multiple trees, but the reproducer is for linux-next... what should we do in such case?

[thazhemadam]

Another approach might be to introduce a new section in the dashboard itself (something like pending review maybe? I don't know).
This would be highly extensible, even in the current context and state of the dashboard.
Currently, there seem to be a lot of bugs that are "open", but which do not get triggered anymore.
If this new section is introduced, bugs which fit atleast one of the following criteria can be filed under it.

• A fix bisection has been found, but has not been approved yet, i.e., nobody has sent in a "#syz fix" mail with that commit as the fix commit
• The bug has had fix bisection sucessfully done in one tree (such as linux-next) and has been auto-closed, and is pending reproducer generation on other trees that the bug seems to have happened on. Even if by chance the reproducer generation fails, one can get a better idea of what fix might work by taking a look at what was done for the other tree, on which the bug no longer occurs.
• And most importantly, bugs for which fix bisection has failed, or hasn't been triggered yet; yet the bug doesn't seem relevant in the present context of the respective tree, and doesn't get triggered anymore.

I feel organizing things this way might not only increase productivity, but also make it easier to navigate, and close bugs faster, and it also saves the hassle of waiting to see if a bug is still occuring or not.

It can be a pain and also time consuming for everyone otherwise, to keep pinging mailing lists and maintainers to look over the bug and see if they know which commit fixed it, so the bug can finally be closed.

[dvyukov]

Thanks for the feedback and ideas, Anant!

A fix bisection has been found, but has not been approved yet

I agree this would be very useful. In some cases these bisection results just get lost.
I can think of a potential extension to this if we implement tagging system (#608). We could add bisection_triage_pending label to bugs when they get fix bisection results. The dashboard section will show open bugs with this label. Then if somebody closes the bug as fixed, we done, it goes away from the dashboard section. However, if the bisection result is incorrect, then a user has a choice of removing the label. Then the bug also goes away from the dashboard section (or at least shown differently).
This will prevent infinite accumulation of bugs with wrong bisection results in that dashboard section, otherwise one will need to look though all of then each time ((O(n^2) complexity :)).

[dvyukov]

The bug has had fix bisection sucessfully done in one tree (such as linux-next) and has been auto-closed, and is pending reproducer generation on other trees that the bug seems to have happened on.

This can't happen. Bugs by design are global. It's not possible to close a bug only in linux-next, but keep it open in other trees. It's a single bug. This issue talks about bugs that happened only on linux-next.

[dvyukov]

And most importantly, bugs for which fix bisection has failed, or hasn't been triggered yet; yet the bug doesn't seem relevant in the present context of the respective tree, and doesn't get triggered anymore.

I think this may be useful as well.
One clarification is that the bug stopped happening, we should simply start fix bisection (which we do already). The situation that we think it's fixed already, but we are not doing fix bisection looks bogus. The predicate for deciding when the bug is fixed may need tuning, though.
So this reduces set of bugs only to: (1) fix bisection failed with a syzkaller bug (something that should not happened ideally), (2) inconclusive fix bisection that arrived to a range of commits due to broken builds/boots.
Maybe we should email case (2) as well? Then this set of bugs becomes almost empty.

[dvyukov]

And this should be moved to a separate issue. Nobody will find it here looking for work to do.

[thazhemadam]

This can't happen. Bugs by design are global. It's not possible to close a bug only in linux-next, but keep it open in other trees. It's a single bug. This issue talks about bugs that happened only on linux-next.

Let me rephrase. I believe I should've been a bit more explicit.

We may have a bug that happens on multiple trees, but the reproducer is for linux-next... what should we do in such case?

In cases like these, when we have reproducers only for one tree (such as the linux-next), and when retested, the bug doesn't happen anymore, it might help to give the bug a label, or provide some sort of indication that says "this bug was fixed on this tree, but is still present on this other tree".
I just felt that it might provide atleast some sort of a frame of reference, that might help people investigating the bug in another tree.

[thazhemadam]

One clarification is that the bug stopped happening, we should simply start fix bisection (which we do already).

Just out of curiousity, are fix bisections automatically started only when a test initiated by syzkaller itself automatically detects that the bug has stopped happening?
Or, are fix bisections also started, when a user sends in a testing request which turns out to be OK (the bug doesn't seem to happen anymore)?
P.S. : When I say user initiated request, I mean to say that the user simply wants to test if the bug is still valid or not. So, it can be understood that the user is NOT attaching any patches to be applied, and the tree the user specifies belongs to the set of trees on which syzkaller detected a bug.

[dvyukov]

This can't happen. Bugs by design are global. It's not possible to close a bug only in linux-next, but keep it open in other trees. It's a single bug. This issue talks about bugs that happened only on linux-next.

Let me rephrase. I believe I should've been a bit more explicit.

We may have a bug that happens on multiple trees, but the reproducer is for linux-next... what should we do in such case?

In cases like these, when we have reproducers only for one tree (such as the linux-next), and when retested, the bug doesn't happen anymore, it might help to give the bug a label, or provide some sort of indication that says "this bug was fixed on this tree, but is still present on this other tree".

We don't try to obtain reproducers on all trees and always do any testing/bisection using only 1 reproducer/tree.
So if we did such testing and the bug appears fixed, we assume it's fixed everywhere. You say "still present on this other tree", but it's not necessary true, I would say the expectation is opposite -- it's also fixed on other trees.

I just felt that it might provide atleast some sort of a frame of reference, that might help people investigating the bug in another tree.

I don't the workflow you are referring to. Why would one investigate it on another tree in this case?

[thazhemadam]

And this should be moved to a separate issue. Nobody will find it here looking for work to do.

Of course, I understand. I was just spitballing an idea as it took form, and I just went with the flow. Would it be alright, if I opened this as a new issue, and referenced this issue as well?

[thazhemadam]

We don't try to obtain reproducers on all trees and always do any testing/bisection using only 1 reproducer/tree.
So if we did such testing and the bug appears fixed, we assume it's fixed everywhere. You say "still present on this other tree", but it's not necessary true, I would say the expectation is opposite -- it's also fixed on other trees.

Ah, alright. I was under the wrongful impression that "fixed on one tree doesn't necessarily mean fixed on another", and that's why I thought someone might feel the need to investigate it on another tree. This clears up my confusions and doubts, and I understand now how that might seem silly, thanks.

[dvyukov]

And this should be moved to a separate issue. Nobody will find it here looking for work to do.

Of course, I understand. I was just spitballing an idea as it took form, and I just went with the flow. Would it be alright, if I opened this as a new issue, and referenced this issue as well?

yes

[dvyukov]

We don't try to obtain reproducers on all trees and always do any testing/bisection using only 1 reproducer/tree.
So if we did such testing and the bug appears fixed, we assume it's fixed everywhere. You say "still present on this other tree", but it's not necessary true, I would say the expectation is opposite -- it's also fixed on other trees.

Ah, alright. I was under the wrongful impression that "fixed on one tree doesn't necessarily mean fixed on another", and that's why I thought someone might feel the need to investigate it on another tree. This clears up my confusions and doubts, and I understand now how that might seem silly, thanks.

Separating statuses per-tree would be very too hard taking into account patch propagation between trees, and possible states and special trees like linux-next. Say, if we don't see a bug on a tree, what does it mean? (1) the bug is not present in that tree? (2) it's already fixed there? (3) the buggy commit did not reach that tree yet? (4) fuzzer was not lucky enough to trigger it on that tree yet? We can't answer this question.

[a-nogikh]

I think this issue is no longer relevant after #3304 was merged.