g.String() produces PGN that WithExpandVariations() cannot re-parse

[demetris-manikas]

Hi.
the description below is written by AI. It does describe my problem clearly I hope.
I have of course run the test locally.
Thanks in advance.

Summary

Game.String() writes consecutive {} {} comment blocks when a move has both
a text comment and command annotations (e.g. [%eval]). When the game contains
nested variations, WithExpandVariations() fails on the re-parsed output.

The library writes what it cannot fully read back.

Steps to reproduce

1. Parse a valid PGN containing text comments and nested variations.
2. Add command annotations to moves using SetCommand("eval", "0.25").
3. Serialize with g.String().
4. Re-parse the output with WithExpandVariations().

Expected behavior

g.String() should produce valid PGN that the library can re-parse.
A move with both a text comment and a command should be serialized as a
single comment block:

6. g3 {A quiet line. [%eval 0.25]}

Per the PGN standard and its extensions:

• The [PGN Specification](http://www.saremba.de/chessgml/standards/pgn/pgn-complete.htm)
  (§8) defines brace comments as { to } and states they "do not nest".

• The [PGN Command Extension](https://www.enpassant.dk/chess/palview/enhancedpgn.htm)
  (§3.2.6) specifies that "a single comment tag may contain multiple embedded
  commands", with this example:

  {[%clk 0:00:07][%eval -6.05] White is toast}
  

• The [annotated PGN spec](https://github.com/mliebelt/pgn-spec-commented/blob/main/pgn-spec-supplement.md)
  reaffirms: "commands can appear at any point inside comments and that there
  may be multiple commands in a single comment."

See also the [PGN Standard repository](https://github.com/fsmosca/PGN-Standard)
for the consolidated specification with supplement.

Actual behavior

g.String() produces two adjacent comment blocks:

6. g3 {A quiet line.} { [%eval 0.25] }

Re-parsing with WithExpandVariations() fails with:

Parser error: no legal move found for position: piece type mismatch (Token: CommentStart, Value: {)

Note: re-parsing without WithExpandVariations() succeeds, and simple games
without nested variations also round-trip correctly. The failure requires the
combination of double comment blocks and nested variations.

Impact

Any workflow that parses a PGN with variations, annotates it with SetCommand,
and writes it back produces output that WithExpandVariations() cannot process.
This breaks round-trip use cases like engine evaluation tools that add [%eval]
annotations to study files containing variations.

Reproducer

package main

import (
	"fmt"
	"os"
	"strings"

	chess "github.com/corentings/chess/v2"
)

const inputPGN = `[Event "Test"]
[Site "?"]
[Date "2024.01.01"]
[Round "?"]
[White "White"]
[Black "Black"]
[Result "*"]

1. e4 {Best by test.} c5 2. Nf3 d6 3. d4 cxd4 4. Nxd4 Nf6 5. Nc3 a6 6. g3 {A quiet line.} (6. Be3 {Active.} e5 (6... e6) 7. Nf3 (7. Nb3 Be6 8. f3 {English Attack.}) 7... Be7 8. Bc4 O-O 9. O-O Be6) 6... e5 (6... e6 7. Bg2 Be7 8. O-O Qc7 9. Be3 O-O) 7. Nde2 (7. Nb3 Be7 8. Bg2 (8. a4 Nc6) 8... O-O 9. O-O b5) (7. Nf3 Be7 8. Bg2 O-O 9. O-O b5) (7. Nf5 d5) 7... Be7 8. Bg2 (8. a4 Nc6) 8... O-O 9. O-O b5 10. Nd5 Nbd7 11. Nec3 Nb6 *
`

func main() {
	origCount := parseExpanded(inputPGN)
	fmt.Printf("Original: %d variations\n", origCount)

	g := parseTree(inputPGN)
	addEvals(g)

	output := g.String()

	newCount := parseExpanded(output)
	fmt.Printf("After round-trip: %d variations\n", newCount)

	if newCount < origCount {
		fmt.Printf("\nFAIL: round-trip lost %d variations\n", origCount-newCount)
		fmt.Println("\ng.String() splits text comments and command annotations")
		fmt.Println("into separate {} blocks. WithExpandVariations() cannot")
		fmt.Println("re-parse the result.")
		os.Stdout.WriteString("\nwriteComments() produces: {A quiet line.}\n")
		os.Stdout.WriteString("writeCommands() appends:  { [%eval 0.25] }\n")
		os.Stdout.WriteString("Combined output:          {A quiet line.} { [%eval 0.25] }\n")
		os.Stdout.WriteString("Expected:                 {A quiet line. [%eval 0.25]}\n")
	}
}

func parseExpanded(pgn string) int {
	s := chess.NewScanner(strings.NewReader(pgn), chess.WithExpandVariations())
	n := 0
	for s.HasNext() {
		if _, err := s.ParseNext(); err == nil {
			n++
		}
	}
	return n
}

func parseTree(pgn string) *chess.Game {
	s := chess.NewScanner(strings.NewReader(pgn))
	s.HasNext()
	g, _ := s.ParseNext()
	return g
}

func addEvals(g *chess.Game) {
	walk(g.GetRootMove(), func(m *chess.Move) {
		m.SetCommand("eval", "0.25")
	})
}

func walk(m *chess.Move, fn func(*chess.Move)) {
	if m == nil {
		return
	}
	fn(m)
	for _, c := range m.Children() {
		walk(c, fn)
	}
}

Output

Original: 10 variations
After round-trip: 0 variations

FAIL: round-trip lost 10 variations

g.String() splits text comments and command annotations
into separate {} blocks. WithExpandVariations() cannot
re-parse the result.

writeComments() produces: {A quiet line.}
writeCommands() appends:  { [%eval 0.25] }
Combined output:          {A quiet line.} { [%eval 0.25] }
Expected:                 {A quiet line. [%eval 0.25]}

[CorentinGS]

Hi,
Thx for the issue. It should be fixed in v2.3.10 :)

[demetris-manikas]

Which way are you going? Fixing the output or the input? My aim would be to make parsing and outputting without any changes produce and identical pgn. This would be an api change though since the Comments method will have to return an array.