Movelist / Tree View now includes color highlights for inaccuracy/mistake/blunder

[yuzisee]

Component

Movelist / Tree View

Implemented in this Pull Request

Some basic color coding for inaccuracies vs. mistakes vs. blunders.

This pull request defines inaccuracy/mistake/blunder based on #242 (comment) → #237 (comment) so that both Stockfish & Lc0 evals can be usable by default in Nibbler for analysis of human games.

Testing

[yuzisee]

FYI I've tested this out quite a bit locally, and stomped out most of the bugs. Let me know if you have any other questions!

[OverLordGoldDragon]

Thanks for your work on this! I've read in #235 that it already works for you - so is it good to go to simply clone your branch and follow the install instructions (Windows 11)? And it supports Stockfish, correct?

[yuzisee]

so is it good to go to simply clone your branch and follow the install instructions

Yes.

• If you already have nibbler installed elsewhere, you can also directly add the files from the git repo's files/src folder directly into your existing nibbler's resources/app folder.
• (or, even easier) here's a zip you can download that runs on windows and has all four branches included: https://www.dropbox.com/scl/fi/qpzh38axktnof6e3clup3/nibbler-2.4.dev-windows.zip?rlkey=wzd7i1fajocd54lx1mtehd0ro

And it supports Stockfish, correct?

I didn't touch anything Leela/Stockfish specific, so hopefully it does. Give it a try and let us know!

[OverLordGoldDragon]

Works great! Tested Stockfish. -- Few things:

1. Annotation (underlining) doesn't work directly with Auto-evaluate line, so one must pass through every move (e.g. by holding down arrow key) when importing a PGN to analyze
2. Can it be made to annotate only my side (White/Black)? I'm rarely interested in opponent's blunders

[yuzisee]

Works great! Tested Stockfish. -- Few things:

1. Annotation (underlining) doesn't work directly with Auto-evaluate line, so one must pass through every move (e.g. by holding down arrow key) when importing a PGN to analyze

Would you mind sharing the PGN file you're seeing issues with? I can take a look and see what's happening.

I assume you're using this button

---

For reference:

Auto-eval is working for me, here's a screenshot run using https://www.dropbox.com/scl/fi/qpzh38axktnof6e3clup3/nibbler-2.4.dev-windows.zip?rlkey=wzd7i1fajocd54lx1mtehd0ro

This is the PGN file from the screenshot
xQc_vs_Franks-is-heres_2023.08.08.pgn.txt

2. Can it be made to annotate only my side (White/Black)? I'm rarely interested in opponent's blunders

I have some ideas about how to do this, but for now let's focus on making sure auto-eval is working for you. We can revisit the one-side visualizations later.

[OverLordGoldDragon]

PGN -- auto example -- after passing through manually

[yuzisee]

For some reason the PGN you attached is different than the one in your screenshot

but I'm definitely getting underlines.

What settings are you using here:

and here

[yuzisee]

@OverLordGoldDragon

Would you mind testing to see if it will work, in the meantime, by enabling Q here

If that works, I can try updating the code so it doesn't require that setting and always highlights/underlines.

[OverLordGoldDragon]

Sorry -- correct PGN

I changed all settings to yours and now it works - the one at which it worked is 1M -> 10M nodes (it was also the last thing that didn't match yours, all else failed), so maybe because it's too fast (I set to 16 cores and my CPU is very vast, it passes several moves per second). Also,

3. Forget all analysis doesn't clear the annotations, I restart Nibbler each time

(Side question: any docs on Nibbler? I can't tell what many of the options do)

[yuzisee]

Forget all analysis doesn't clear the annotations, I restart Nibbler each time

Oh, thanks for catching that! I probably need to add a line or two to

nibbler/files/src/renderer/51_node.js

Line 479 in 004e66b

function __clean_tree(node) {

Let me take a look at that.

(Side question: any docs on Nibbler? I can't tell what many of the options do)

Not sure, but @rooklift I'm happy to either start some docs or refresh anything that might already exist. Happy to contribute some cleanup here and there or expand any existing documentation if helpful.

@OverLordGoldDragon in the meantime, if there are any specific options you'd like to learn more about I can probably read the source code and give you an answer 🙂

[OverLordGoldDragon]

Well in our "quest" to mimic Lichess I think it crucial to replicate the evaluation graph - @ rooklift's given me a pointer but I've not had a chance to dig into it.

I've also noticed @ rooklift's comment on another thread which I think belongs in README - i.e. if using classical engines, ensure MultiPV=1, and set Threads and Hash (from Stockfish forums I hear recommended is <70% of RAM).

[yuzisee]

Discussion, continuing from #242 (comment) and building on #242 (comment)

• If we follow the Lichess convention of "blunder" being a 25%+ change in winpct, Lichess is essentially arguing that a blunder is 2.98 centipawns from a drawn position. For Stockfish, a blunder is 1.0 centipawns from a drawn position on move 32.
• If we follow the Lichess convention of "mistake" being a 10%+ change in winpct, Lichess considers 1.1 centipaws from a drawn position to be a mistake. Stockfish would consider 0.67 centipawns to be a mistake from a drawn position on move 32.
• If we follow the Lichess convention of "inaccuracy" being a 5%+ change in winpct, any winrate past 95% can no longer produce inaccuracies for your opponent because the winpct can't increase by 5% in a single move without becoming more than 100%. Therefore:
  • An eval of ±1.65 is the point at which Stockfish evals could be considered "virtually the same" according to the Stockfish WDL model.
  • An eval of ±7.99 is the point at which Stockfish evals could be considered "virtually the same" according to Lichess live player data.

formula	−7.99	−2.98	−1.65	+1.65	+2.98	+7.99
SF v16 @ move 0			98.8% B : 1.2% W	1.2% B : 98.8% W
SF v16 @ move 32			98.6% B : 1.4% W	1.4% B : 98.6% W
SF v16 @ move ≥120			95% B : 5% W	5% B : 95% W
Lichess 2023	95% B	75% B	64.7% B	64.7% W	75% W	95% W

Conclusion

The suggestions in #242 (comment) were to use 0.25 & 0.5, but they would probably be too small in practice.

formula	eval −0.5	−0.25	+0.25	+0.5
Lichess 2023	54.6% B : 45.4% W	52.3% B : 47.7% W	47.7% B : 52.3% W	45.4% B : 54.6% W
SF v16 @ move 0	54.7% B : 45.3% W	51.3% B : 48.7% W	48.7% B : 51.3% W	45.3% B : 54.7% W
SF v16 @ move 32	53.1% B : 46.9% W	50.8% B : 49.2% W	49.2% B : 50.8% W	46.9% B : 53.1% W
SF v16 @ move ≥120	52.9% B : 47.1% W	50.9% B : 49.1% W	49.1% B : 50.9% W	47.1% B : 52.9% W

Maybe:

• ±0.5 pawneval = inaccuracy
• ±1.0 pawneval = mistake
• ±3.0 pawneval = blunder

within the stipulation that all evals larger than ±2.5 are to be considered virtually the same.

[yuzisee]

Rebased onto latest master, and updated after #244

---

For posterity, here's how the other platforms handle it:

chess.com

Note that chess.com also has a "Missed Win" annotation that is more severe than "Blunder".

Lichess

The Lichess rules are slightly different, and are based instead on win percentage change (±0.1, ±0.2, and ±0.3/0.5 — NOTE: Lichess defines cpWinningChances ranging from −1.0 to +1.0). Longer discussion is here #242 (comment)

[yuzisee]

September 25, 2023:

• Tested working on latest master
• Screenshot updated above