v0.14.0 — Go grammar compiled by grammargen; -54% warm / -61% cold memory

Highlights

gotreesitter's own pathological Go files — parser.go, parser_reduce.go, parser_test.go, query_test.go, and the parser_result_* set — now parse cleanly (HasError=false at root). The old tree-sitter-go C tables wrapped parser_reduce.go in a root-level ERROR because of a dead-end state where } had no action after certain nested switch/case/if patterns.

Memory footprint on the representative BenchmarkSelfParseWarmReuse (six pathological gotreesitter root files, 5-iter warm bench, Docker 4 g / 4 cpus):

mode	pre-0.14.0	0.14.0	delta
cold (fresh Parser per iter)	574 MB/op	225 MB/op	-60.8 %
warm (one Parser reused)	498 MB/op	229 MB/op	-54.0 %
warm + GC drain between rounds	522 MB/op	252 MB/op	-51.7 %

Warm-reuse throughput ~10 % higher. 206-grammar parity stays green under GTS_PARITY_MODE=exhaustive.

Rolled-up changes

• Go grammar now ships as a grammargen-compiled blob (PR #35). Our pure-Go LALR(1) + LR(1) state-splitting compiler produces a different state layout that sidesteps the tree-sitter-go dead-end. cmd/emit_grammargen_go_blob lets us regenerate as grammargen evolves. The custom GoTokenSource is no longer registered by default — the grammargen blob's DFA parses Go cleanly on its own; GoTokenSource stays available via the public API for callers carrying their own ts2go Go blob.
• Go initial GLR stack cap raised from 2 to 32 (PR #36). The previous cap=2 was chosen for the ts2go Go blob to avoid exponential blowup on large files; grammargen's blob has a different conflict profile where cap=2 was instead triggering a guaranteed two-retry cycle on every non-trivial file. Retry invocations across the self-parse benchmark drop from 8 to 0.
• Parser pool aliasing on recovery token sources (PR #30 by @rasmus-theca). Recovery reparsing was acquiring a pooled dfaTokenSource while the outer parse still held one; fixed via a non-pooled variant with noPool: true.
• Zig grammar migrated from maxxnino/tree-sitter-zig (inactive since 2024-10) to tree-sitter-grammars/tree-sitter-zig (PR #32, addresses #31). PascalCase → snake_case rename; 28 % smaller blob. Three upstream #lua-match? highlight predicates rewritten as #match?. Review-follow-up commit fixes four gemini-flagged items.
• Arena initial-sizing heuristic (PR #33): sourceLen × 4 → sourceLen / 4. The old formula over-allocated 10-16× for Go; the adaptive hint handles subsequent parses.
• Arena retention ceiling preserved across resets (PR #33). Warm-reuse workloads no longer re-reallocate the primary slab every parse.
• Retry path releases losing candidate-tree arenas eagerly (PR #34). Arenas no longer sit in GC limbo between retry attempts.
• Tier-1 grammar lock refresh (PR #26). 10 tier-1 grammars pinned to current upstream tips: dart, elixir, erlang, kotlin, ocaml, php, ruby, rust, scala, swift.
• DrainArenaPools() + releaseNodeRefs on reuseCursor/reuseScratch (PR #25 by @vdergachev). Call after a large batch scan to allow GC reclamation.
• Dead GLR helpers removed (PR #29 by @lars-l).
• BenchmarkSelfParse / BenchmarkSelfParseWarmReuse added as regression guards.

Upgrade notes

• The Go grammar blob is structurally equivalent to the old ts2go blob (all parity tests pass) but compiled by a different LR generator. Any code that hard-codes tree-sitter-go's state IDs or symbol IDs will need adjustment — grammargen's state layout is different (1910 states). Node names, node kinds, and the public parse-tree shape are unchanged.
• grammars.NewGoTokenSource / grammars.NewGoTokenSourceOrEOF are still callable as public API. They are no longer registered as the default token source for Go; if you depend on that registration, re-register via the public registerTokenSourceFactory path with your own ts2go-compiled Go blob.

Credits

• @vdergachev — PR #25
• @lars-l — PR #29
• @rasmus-theca — PR #30