v0.6.0

• Added --with-counterexamples to golr parser, which shows for every listed conflict either an input with two derivations, proving the grammar ambiguous, or two derivations that share the input up to the conflict. The search is limited by --counterexample-time-limit and --counterexample-total-time-limit, and available as pkg/parsergen/counterexample.
• Added golr lsp, a language server for GoLR grammar files with go to definition, find references, rename, code completion, semantic highlighting, reference counts and formatting, together with editor setups for Neovim, Vim and Emacs in ide/.
• BREAKING: The conflict report writers moved from pkg/parsergen/conflict to the new pkg/parsergen/report, with ReportConfig renamed to report.Config, WriteUnresolvedConflictReport takes the grammar instead of the error, and both take the counterexamples of the conflicts. ConflictReport, ConflictReportEntry and ConflictReportReduction are no longer exported.
• BREAKING: The GoLR cores return the parser tables together with the error on unresolved conflicts, still holding the conflicting actions, and on a failure due to FailOnWarnings. UnresolvedConflictError no longer carries a report, and its message is a single line naming the terminal and the state.
• Updated the mimimum Go version from 1.25 to 1.26 and updated the Go toolchain to 1.27.
• Fixed golr fmt not aligning scanner rules when a blank line separated a rule name from its value, so formatting the output again changed it.
• Fixed a regular expression which used the same fragment twice, like /{F}b|{F}/, changing the first use of the fragment.
• Fixed golr scanner panicking on a repetition with a maximum of zero like /a{0}/, which is now reported as an error.
• Fixed golr panicking on a terminal or fragment which is an empty string, which is now reported as an error.
• Fixed golr convert writing a regular expression which is a plain literal as a string, which added an alias that could collide with other terminals.
• Fixed golr convert writing / and [ inside a character class without escaping them, which gave a regular expression that did not read back.
• Fixed golr convert producing invalid GoLR for Bison grammars with terminal names containing . or -, with undeclared string literal tokens, with renamed symbols colliding with terminals, or with the alias "".
• Fixed the Bison frontend accepting a rule for a symbol declared as a token.
• Fixed golr convert not applying the Bison specific conversions when reading the grammar from stdin.
• Fixed golr scanner panicking on a scanner without rules, or when no rule matches any input, like a single rule with the empty character class [].
• Fixed a repetition with a minimum of zero, like /ba{0,1}/, matching the empty string when it was not at the start of the regular expression.
• Fixed golr scanner panicking on JSON or YAML rules with an invalid regular expression or a node without the value of its kind, which are now reported as errors.
• Fixed golr parser panicking on a JSON or YAML grammar with a symbol index out of bounds, which is now reported as an error.
• Fixed the GoLR grammar writer, and with it golr convert, regrouping the productions of a nonterminal defined in several places, which could change how reduce/reduce conflicts are resolved.
• BREAKING: golr convert and convert.BisonGrammar2GoLR now fail on Bison terminals which share an alias, which GoLR does not allow, instead of writing a grammar file GoLR cannot read. BisonGrammar2GoLR returns an error for that.
• Fixed the Bison frontend accepting string literals, translatable strings and character literals which span over a newline, which GNU Bison rejects.
• Fixed golr convert writing the error symbol of a Bison precedence declaration like %left error as $error, which made the converted grammar unreadable. It is written as @error now.
• BREAKING: GoLR string literals now support the escape sequences " and \ and reject every other backslash escape, so a string literal can end in a backslash.
• Fixed the Bison converter copying the escape sequences of Bison strings and character literals verbatim instead of translating them to GoLR strings.
• Fixed the Bison converter giving two string literal tokens the same name, like "_" and " ".
• Fixed a crash on Bison grammars whose rules section holds declarations only, which are now refused like GNU Bison does.
• Fixed a crash on character classes with a range ending in a UTF-16 surrogate code point.
• Fixed a crash on concatenations and alternations with missing children in JSON and YAML scanner rules.
• Fixed DFA minimization taking quadratic time, which made some regular expressions take minutes to build.
• Fixed crashes on malformed YAML grammars and scanner rules and YAML output which read back differently, e.g. strings like "\n" or ".inf", by switching from goccy/go-yaml to go.yaml.in/yaml/v3.

v0.5.0

• Conflicts decided by precedence and associativity declarations are no longer reported. The conflict summary and --verbose listing of golr parser, and the conflicts returned by the GrammarToParser functions of the GoLR cores, now only cover conflicts resolved by shift over reduce or by the earliest production, plus unresolved ones.
• Fixed the Bison frontend reading a string literal which was not declared as a token alias, like "+", as a nonterminal without productions instead of as a terminal.
• Adjusted conflict reporting to be stable. This allows conflict reports to be diffed reliably, even with the grammar being modified.
• Unresolved conflicts are now reported in full.
• Added --with-state-number to golr parser, which adds the state number to every state of the conflict report.
• BREAKING: Parse tree nodes carry the byte offset and length of the source they cover instead of a lexeme. Scanners resolve these through the new Position and Text methods, which replace Line and Column.
• BREAKING: Custom token sources have to provide Position and Text. A token they insert needs an empty lexeme at the offset where it is inserted.
• BREAKING: C scanners allocate a line table on the first position lookup. Call _scanner_free once a scanner is done with, or the table leaks.
• BREAKING: The Rust ParseNode no longer has a lifetime parameter, and the Rust Scanner is no longer Sync.
• Semantic errors in GoLR and GNU Bison grammar files, like undeclared terminals or duplicate declarations, now report the file, line and column they occur at.
• BREAKING: The column reported by the Position method of generated scanners now counts UTF-8 characters instead of bytes, and a tab counts as one column. Use the byte offset to get at the bytes of the source.
• Added --fail-on-sr-conflicts, --fail-on-rr-conflicts and --fail-on-conflicts to golr parser, which make the run fail on a shift/reduce or reduce/reduce conflict that precedence and associativity do not resolve.
• The conflict summary of golr parser now counts resolved and unresolved conflicts separately, and a conflict decided by shift over reduce no longer also counts as a reduce/reduce conflict.
• The IntelliJ and vscode plugin now support Go to Definition and Find Usages on terminals referenced by their string alias, like "+", and counts them in the usage count of the terminal.
• The IntelliJ and vscode plugins can rename the string alias of a terminal, which renames the string in the terminal declaration and every reference by that string.
• The IntelliJ and vscode plugins now completes keywords like @parser, @name and @error, offering those valid at the caret, and only offer symbol names where a symbol can be referenced.
• BREAKING: The parser generator now fails on nonterminals which do not derive any finite string, including nonterminals which are used but never defined.
• BREAKING: GrammarToParser of every core and Grammar.Validate now return a slice of warnings.
• The parser generator now warns about nonterminals which are unreachable from the start nonterminal.
• The parser generator now warns about productions which are never reduced after conflict resolution.
• Added --fail-on-warnings to golr parser and core.FailOnWarnings, which make the run fail on warnings.
• Syntax errors of generated parsers now list the expected tokens and name tokens by their string alias, like: unexpected "+", expecting NUMBER.

v0.4.0

• Fixed the scanner walking its line and column counters past the end of the source when Reset was given an offset beyond it. The offset is now clamped, as it already was in the C and C++ backends.
• BREAKING: Aligned the generated scanners and parsers across all language backends.
• Added a Kotlin backend for the scanner and parser generator, selected with --backend kotlin and configured with --backend-kotlin-package-name.
• Improved error recovery for malformed GoLR grammar files.
• The GoLR grammar frontend now rejects regex and string literals which span multiple lines, so a missing closing delimiter no longer consumes the rest of the file.
• Added the @name annotation to the GoLR grammar frontend, letting a production alternative be given an explicit name instead of the auto-generated _.
• Generated parsers now expose every production as a named Production constant and set it on the ParseNode/Node each reduction produces, so a tree walk can switch on production identity instead of counting children.
• Generated parsers can emit a text trace of every action they take - each shift, reduce, accept and error recovery step - through an optional trace hook set on the parser instance, as a debugging aid for understanding an unexpected parse.
• Fixed golr fmt discarding all comments from GoLR grammar files. It now preserves comments and blank lines, and no longer fails on a grammar file with broken syntax.

v0.3.0

• Added a JavaScript scanner backend.
• Added a JavaScript parser generator backend, selected with --backend javascript. The generated ECMAScript module imports the token constants from the generated scanner, whose module specifier can be set with --backend-javascript-scanner-module.
• Added a TypeScript scanner generator backend.
• Added a TypeScript parser generator backend.
• Added a C# scanner generator backend.
• Added a C# parser generator backend.
• Replaced the Java scanner backend with a table driven one.
• Added a Java parser generator backend, selected with --backend java.
• Replaced the Rust scanner backend with a table driven one.
• Added a Rust parser generator backend, selected with --backend rust.
• Added a Python scanner generator backend, selected with --backend python.
• Added a Python parser generator backend, selected with --backend python.
• Added a C++ scanner generator backend, selected with --backend cpp.
• Added a C++ parser generator backend, selected with --backend cpp.
• Fixed the generated scanners consuming one byte too many when a match failed after the automaton had already consumed bytes. The byte which ended the failed match now starts the next token instead of being swallowed by the invalid token, which had corrupted the token following the error.
• Fixed the generated parser rejecting an empty input which the grammar accepts. The end of input is now a token the parse table decides on like any other.
• Fixed Symbol.String of the generated Go parser printing the index of a symbol instead of its name.
• BREAKING: The generated Go parser now spells the augmented start symbol AcceptNonterminal instead of Nonterminal_accept, which is the name every other language backend already uses.
• BREAKING: --backend go now writes the table driven scanner and parser, which is smaller and faster than the directly coded one it replaces. The directly coded Go backend is gone, and with it the names go-direct and go-table.
• BREAKING: The public packages pkg/parsergen/backend/golangtable and pkg/scannergen/backend/golangtable are now pkg/parsergen/backend/golang and pkg/scannergen/backend/golang, taking the place of the re-exports of the directly coded backend.
• Added a C scanner generator backend, selected with --backend c.
• Added a C parser generator backend, selected with --backend c.

v0.2.0

• The IELR(1) parser core is now a native Go implementation and is used by default. GNU Bison is no longer required to generate a parser. The Bison backed core is still available as "ielr1-bison".
• The LALR(1) and LR(1) parser cores are now native Go implementations as well and are used by default. The Bison backed cores are still available as "lalr1-bison" and "lr1-bison".
• Generated parsers now recover from syntax errors. Mark the places to resume at with the error symbol (@error in GoLR grammars, error in GNU Bison grammars) and the parser reports several errors in one run instead of stopping at the first one.
• The VS Code and IntelliJ plugins now support the @error token.
• The parser command now reports shift/reduce and reduce/reduce conflicts on stderr. The new --verbose flag additionally lists every conflict which was resolved automatically.
• Generated parser tables are smaller, because reductions are compacted into a default reduction and unreachable states are removed.
• The LALR(1) and IELR(1) cores now report an error when a grammar needs more states than the parser tables can address. Previously only the LR(1) core did.
• Parser generation for large grammars is significantly faster and needs less memory.
• The released binaries are built without debug assertions, which makes parser generation noticeably faster.
• The parsers and scanners generated by GoLR are exempt from the Apache License, Version 2.0. You can license the generated files as you choose.
• Fixed %nonassoc not rejecting sentences which it should reject.
• Fixed the Go backend generating broken accept actions, which could result in generated code which does not compile.
• Fixed terminals declared with %precedence being dropped when handing the grammar to the Bison backed cores, which silently changed how conflicts were resolved.
• Fixed the precedence of a production getting lost while the grammar is augmented.
• BREAKING: GrammarToParser of the pkg/parsergen/core/... packages now returns the list of conflicts as an additional result. Call sites need to be updated.
• Added the golr selftest command, which checks the IELR(1) parser core against a canonical LR(1) oracle on randomly generated grammars across all CPU cores for as long as you let it run.
• Fixed index out of range panics in the Bison frontend scanner when a grammar file ends inside a tag, prologue or braced code block.
• Fixed the Bison frontend not recognizing the %lex-param and %parse-param directives.
• Fixed the Bison frontend scanner swallowing input after char literals or strings ending in an escaped backslash, like '\'.
• Fixed the Bison frontend ignoring char literal token declarations like %token '\' and misnaming escaped char literals: escape sequences are now decoded, so different spellings of the same character resolve to the same terminal.
• Added the scanner backend go-table, which writes the scanner as lookup tables instead of as code.
• Added the new parser backend 'go-table' which writes the parser as lookup tables instead of as control flow. The existing Go backend is now also available as 'go-direct', with 'go' remaining an alias for it.

v0.1.0

Changelog

• Initial release