Feature Request: Natural language explanations for FEN positions using Maia 3 / Chessformer

[LyanMiller]

Squares as Tokens, Geometric Attention Bias (GAB), Dot-Product Attention, Source-Destination Policy Head, and Sparse Transcoders allow for clear identification of where the model is "looking." Is there any intention to create a tool that explains FEN chess positions in natural language (NLP) using Maia-3 (Maia 2600)?

[daniel-monroe]

Autonomously generating natural language explanations for chess is an active research area and isn't something that our models support unfortunately. Another breakthrough is likely needed before that is possible.

[LyanMiller]

I understand. Even so, I reopened it here just to say how grateful I am. I used Maia 3 to show where the model is paying attention, and it's interesting that you can already understand the best moves it chooses based on that.

In this position, you might think that Maia wants to capture the bishop on a7, but in reality he seems to be interested in the bishop on c4 first. Then, he seems to plan to play pawn b6 afterwards to capture the remaining bishop.

Note that playing pawn b6 immediately to trap the bishop would be checkmate with two bishops, so he wants to get rid of the light-squared bishop first.

I can't wait to see what future possibilities are available with this project. Another axample of a heat map, the stockfish suggests a3 straight, but the Maia 2600 seems to be trying to trap the knight on the next move.

[wickeduk]

What might be useful here is that if maia can already correctly identify features of interest, then maybe working with a strong player one can try to reconcile what the features show with what the player thinks they mean. If this is done over enough positions or positions of a certain type then it might be possible to assign some kind of probability (based on other features) to what a specific feature means in a position, at least enough that with other things a human can put 2 and 2 together to work the rest out. So for example imposing oneself with an isolated pawn (already lots of material here) could mean that the player is wishing to attack, or that it could be to open some files for the rooks, or provide a knight outpost to take control of another square to penetrate... A central pawn move could be to release some pieces, control some squares, push opponent's pieces back or restrict them, offer a sacrifice to open some lines.. This again can be studied at different levels to make a more human explanation and be very useful to compare what players like the user are thinking versus how stronger players see it (and also weaker ones for coaching purposes).

This is probably the easier bit, the real problem in this space is not what the moves might mean, but how they form some kind of narrative and plan which might be over a few moves, or based on knowledge about the types of positions the opening involves (and is more the domain of expertise by grandmasters). Sometimes plans can be worked out by giving free (null) moves in non-critical positions (i.e. don't have to capture back or deal with a check) and seeing where the pieces get placed and then gradually adding in the other player's play. Similarly what if a move is/isn't played and comparing the analysed lines until the feature appears which differentiates them can explain why a particular move is being played/suggested by a strong engine.

Some 'research' into how players of different levels think was done in this book: https://chess.co.uk/products/think-like-a-super-gm-michael-adams-philip-hurtado and may be useful to understand how different players of strength see positions and the narratives players give.