Bisectable

Aleks-Daniel Jakimenko-Aleksejev edited this page Nov 18, 2016 · 5 revisions

This bot is meant to help you find when something got broken. If you want to know if something has ever worked use Committable instead.

Defaults to old=2015.12 and new=HEAD. Note that there are builds available up to 2015.07, so you have to use old=2015.07 if you really want to go that far.

You can find some extra examples in tests: bisectable.t

Usage examples

Bisect by exit code

<AlexDaniel> bisect: exit 1 if (^∞).grep({ last })[5] // 0 == 4 # RT 128181
<bisectable6> AlexDaniel, bisect log: https://gist.github.com/c3317662a9619214a587d1781fa495eb
<bisectable6> AlexDaniel, (2016-03-18) https://github.com/rakudo/rakudo/commit/6d120ca

In theory, this is what you are supposed to do. That is, you should exit 0 if this revision is OK and exit with any other code if it is not. Use exit 125 to skip a commit. However, over the weeks this bot was getting smarter and smarter, and nowadays it probably knows what you want better than you do. In other words, feel free to throw any piece of code into it and see how it goes. Inspect the log to make sure that you are getting what you were searching for.

Keep in mind that it is mostly a pure binary search, so do not expect it to figure out flappers or any other complex cases.

Bisect by output

<AlexDaniel> bisect: say (^∞).grep({ last })[5] # same but without proper exit codes
<bisectable6> AlexDaniel, Exit code is 0 on both starting points (good=2015.12 bad=575dda1), bisecting by using the output
<bisectable6> AlexDaniel, bisect log: https://gist.github.com/f0e146242faf1ec01602eb6a66337e3a
<bisectable6> AlexDaniel, (2016-03-18) https://github.com/rakudo/rakudo/commit/6d120ca

Inverted exit code

<AlexDaniel> bisect: class A { has $.wut = [] }; my $a = A.new; $a.wut = [1,2,3]
<bisectable6> AlexDaniel, For the given starting points (good=2015.12 bad=575dda1), exit code on a ‘good’ revision is 1 (which is bad), bisecting with inverted logic
<bisectable6> AlexDaniel, bisect log: https://gist.github.com/831baa61429add18413a62ab22e974b0
<bisectable6> AlexDaniel, (2016-03-02) https://github.com/rakudo/rakudo/commit/fdd37a9

Nothing to bisect

<AlexDaniel> bisect: say ‘hello world’; exit 42
<bisectable6> AlexDaniel, On both starting points (good=2015.12 bad=575dda1) the exit code is 42 and the output is identical as well
<bisectable6> AlexDaniel, Output on both points: hello world

Specify good and bad commits

Use good=… and bad=… in front of your query. You can use = or space as a delimiter, it does not matter.

<AlexDaniel> bisect: good=2016.02 bad 2016.03 say (^∞).grep({ last })[5]
<bisectable6> AlexDaniel, Exit code is 0 on both starting points (good=2016.02 bad=2016.03), bisecting by using the output
<bisectable6> AlexDaniel, bisect log: https://gist.github.com/87568a8cf839f8057968340b0ce5e4b8
<bisectable6> AlexDaniel, (2016-03-18) https://github.com/rakudo/rakudo/commit/6d120ca

Sometimes people confuse what is good and what is bad. If that happens, you are going to see this:

<AlexDaniel> bisect: good=2016.03 bad 2016.02 say (^∞).grep({ last })[5] # swapped good and bad revisions
<bisectable6> AlexDaniel, Exit code is 0 on both starting points (good=2016.03 bad=2016.02), bisecting by using the output
<bisectable6> AlexDaniel, bisect log: https://gist.github.com/bb672c28b8b9744f578b94259a2a7557
<bisectable6> AlexDaniel, bisect init failure

If you take a look at the log you will see this:

Some good revs are not ancestor of the bad rev.
git bisect cannot work properly in this case.
Maybe you mistake good and bad revs?

This error message comes from git.

Basically, good is always old and bad is always new.

Special characters

You can use characters as newlines. In other words, the behavior is identical to Committable#special-characters.

<AlexDaniel> bisect: # newline test ␤ say ‘hello world’; exit 42
<bisectable6> AlexDaniel, On both starting points (good=2015.12 bad=575dda1) the exit code is 42 and the output is identical as well
<bisectable6> AlexDaniel, Output on both points: hello world

URLs

It also supports URLs, but note that you have to provide a direct link to the “raw” version:

<AlexDaniel> bisectable6: https://gist.githubusercontent.com/atweiden/9d1dfe825ade18a7db54d8e0733ca2e4/raw/16b7c39cfd12fef6eb74851c872097e0c655cff3/pkginfo.p6
<bisectable6> AlexDaniel, Successfully fetched the code from the provided URL.
<bisectable6> AlexDaniel, Exit code is 1 on both starting points (good=2015.12 bad=575dda1), bisecting by using the output
<bisectable6> AlexDaniel, bisect log: https://gist.github.com/81def212f00afc7dfcd51dde863ed23f
<bisectable6> AlexDaniel, (2016-02-18) https://github.com/rakudo/rakudo/commit/9983c2c

More examples

<AlexDaniel> bisect: bad=d3acb938 try { NaN.Rat == NaN; exit 0 }; exit 1
<bisectable6> AlexDaniel, bisect log: https://gist.github.com/53402fecc6601d9f92f611f17d74fe53
<bisectable6> AlexDaniel, (2016-05-02) https://github.com/rakudo/rakudo/commit/e2f1fa7
<AlexDaniel> bisect: for ‘q b c d’.words -> $a, $b { }; CATCH { exit 0 }; exit 1
<bisectable6> AlexDaniel, bisect log: https://gist.github.com/7ef27b210b3f86b06d1e0f703a91c708
<bisectable6> AlexDaniel, (2016-03-01) https://github.com/rakudo/rakudo/commit/1b6c901
<AlexDaniel> bisectable6: help
<bisectable6> AlexDaniel, Like this: bisectable6: good=2015.12 bad=HEAD exit 1 if (^∞).grep({ last })[5] // 0 == 4 # RT128181
<AlexDaniel> bisectable6: http://github.org/sntoheausnteoahuseoau
<bisectable6> AlexDaniel, It looks like a URL, but for some reason I cannot download it (HTTP status line is 404 Not Found).
<AlexDaniel> bisect: https://gist.github.com/atweiden/9d1dfe825ade18a7db54d8e0733ca2e4
<bisectable6> AlexDaniel, It looks like a URL, but mime type is ‘text/html; charset=utf-8’ while I was expecting ‘text/plain; charset=utf-8’. I can only understand raw links, sorry.