bisect bot

[timotheecour]

slightly off topic (so feel free to close) but since you have this repo that helps with tooling around issues, just wanted to suggest an idea I've been having (timotheecour/Nim#98 (comment)):

the goal is to automate bisection in github issues to:

• find whether an issue is a regression or an always existing bug
• find which commit/PR introduced the regression
• further reduce

this would work using the following ingredients:

• github issue trigger that would run a dedicated CI pipeline (see https://help.github.com/en/actions/reference/events-that-trigger-workflows#issue-comment-event-issue_comment)
• the pipeline would search for a specific section in issue (mentioned in issue template), or simply parsing existing issue template
• the pipeline would run git bisect to find the 1st commit introducing the bug (if any)
• the git bisect output would be automatically inserted as metadata on issue (there are API's for that)
• the author who sent out offending commit gets notified automatically

knowing which commit introduced an issue makes life a bit easier.

This could be usable as a cmdline/library (without github integration) as well as a CI pipeline.
That kind of tool exists in some software companies and is quite useful.

note

• see regression (0.19.6 => 0.20): is broken with generic types nim-lang/Nim#13066 (comment) to automatically find how to build nim at a given revision (ie, finding csources HEAD at that point)

draft below

In my ideal world this would work:

• user submits a bug, following (possibly revised) issue template; a good bug will be minimized, include all relevant context (via your tool, but IMO we can improve it a bit to reduce friction, eg using single cmd, we can discuss this separately)
• nim-bisectbot (bisect bot #8) takes over in the background:
• it gets notified via github API
• parses issue content to extract info it needs (example code)
• verifies validity of the bug otherwise it notifies bug author
• runs git bisect to find 1st offending commit
• notifies author of offending commit
• finds which OS the bug appears on (amongst all CI machines), to check if it's machine specific
• reduces the bug as much as possible, using nim-dustmite (see [TODO] nim-dustmite source code minimization tool for automated bug reduction nim-lang/Nim#8276; it works amazingly well in D, I've saved a ton of time in the past thanks to D's dustmite)

[juancarlospaco]

I have the same idea for few months now.

[timotheecour]

@juancarlospaco btw are you on Nim's gitter ? couldn't find your login name (also tried to contact your via email..)

[juancarlospaco]

Nope.

[juancarlospaco]

For historical log:
https://github.com/juancarlospaco/nim_packages_security_audit#fully-automated-nim-packages-security-audit

[timotheecour]

see nim-lang/Nim#18119