Release 0.6.2

[siy]

Patch release 0.6.2.

Fixed

• Shift operators (<</>>/>>>) in field/local-var initializer context — inline expansion of DFA-skipped lexer rules; selfhost fixture (40K LOC) now parses with 0 diagnostics; both Java25SelfHostDiagTest assertions re-enabled.
• Per-iteration %whitespace tokenization — folded %whitespace <- (...)* emits per-kind trivia tokens; canonical grammar split workaround reverted; 2 trivia tests re-enabled.

Added

• Loud fromGrammar failure (SkippedRuleReferenced) — broken parsers are no longer generated silently.
• ShiftOperatorInlineExpansionTest (16 regression tests).

Validation: 1420 tests / 0 failures / 2 pre-existing skips; JMH flat (reference 2.68 ms, selfhost 71.9 ms); release profile builds sources+javadoc; docs audited and updated.

Summary by CodeRabbit

• Bug Fixes

  • Resolved shift operators (<<, >>, >>>) failing at compilation level in field/local-variable initializers.
  • Fixed whitespace tokenization to correctly preserve per-iteration trivia classification, ensuring proper token emission.

• Improvements

  • Enhanced error reporting to fail loudly when a referenced lexer fails to compile.

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 0ce92413-9f0a-4548-ac2f-c1b7132baf64

📥 Commits

Reviewing files that changed from the base of the PR and between 63ea7d1 and f3c6276.

📒 Files selected for processing (22)

• CHANGELOG.md
• README.md
• docs/ARCHITECTURE-0.6.0.md
• docs/GRAMMAR-DSL.md
• docs/HANDOVER.md
• docs/VISITOR-TUTORIAL.md
• peglib-core/pom.xml
• peglib-core/src/main/java/org/pragmatica/peg/v6/PegParser.java
• peglib-core/src/main/java/org/pragmatica/peg/v6/generator/LexerGenerator.java
• peglib-core/src/main/java/org/pragmatica/peg/v6/generator/ParserGenerator.java
• peglib-core/src/main/java/org/pragmatica/peg/v6/lexer/DfaBuilder.java
• peglib-core/src/main/java/org/pragmatica/peg/v6/lexer/LexerEngine.java
• peglib-core/src/test/java/org/pragmatica/peg/v6/diagnostic/Java25SelfHostDiagTest.java
• peglib-core/src/test/java/org/pragmatica/peg/v6/generator/ShiftOperatorInlineExpansionTest.java
• peglib-core/src/test/java/org/pragmatica/peg/v6/lexer/TriviaClassificationTest.java
• peglib-core/src/test/resources/java25.peg
• peglib-formatter/pom.xml
• peglib-incremental/pom.xml
• peglib-maven-plugin/pom.xml
• peglib-playground/pom.xml
• peglib-runtime/pom.xml
• pom.xml

---

📝 Walkthrough

Walkthrough

Release 0.6.2 resolves shift-operator compilation failures and per-iteration whitespace trivia tokenization. The DfaBuilder detects inline-expandable shift-operator patterns and refactors whitespace absorption into per-alternative trivia kinds. ParserGenerator emits multi-token inline expansions and loud failures for skipped lexer rules. Grammar adopts folded whitespace form with comprehensive regression tests validating shift operators and trivia parity.

Changes

Shift operator inline expansion and per-iteration whitespace tokenization

Layer / File(s)	Summary
DFA inline expansion detection and whitespace absorption refactor peglib-core/src/main/java/org/pragmatica/peg/v6/lexer/DfaBuilder.java	DfaBuilder introduces InlineExpansion record and extends TokenKindAssignment with inline-expansion metadata. The detectInlineExpansions method identifies shift-operator rules matching literal+ !literal patterns, allocates inline kinds for constituents, and removes fused rules from standalone kind map. The absorbWhitespace method replaces whole-body compilation with per-alternative trivia-kind classification, ensuring non-empty matches and fallback behavior.
Parser code generation for inline expansions and skipped rule validation peglib-core/src/main/java/org/pragmatica/peg/v6/generator/ParserGenerator.java	ParserGenerator adds a new 6-arg generate overload accepting skippedReasons map. The sealed ParserGenerationError expands with SkippedRuleReferenced record for rejecting references to skipped lexer rules. Renderer stores skippedReasons and passes it to emitReference for LEXER rules. The emitInlineExpansion helper generates code for multi-token lexer expansion with adjacency checking and negative-lookahead verification.
Lexer trivia refinement for comment classification peglib-core/src/main/java/org/pragmatica/peg/v6/generator/LexerGenerator.java, peglib-core/src/main/java/org/pragmatica/peg/v6/lexer/LexerEngine.java	LexerGenerator widens the generated-code condition triggering content-based trivia refinement to include structurally-classified LINE_COMMENT and BLOCK_COMMENT tokens. LexerEngine.lex extends entry conditions and Javadoc to document expanded scope: comment-prefix reclassification now distinguishes /// from // and /** from /* even when pre-classified.
Parser integration and grammar update to folded whitespace peglib-core/src/main/java/org/pragmatica/peg/v6/PegParser.java, peglib-core/src/test/resources/java25.peg	PegParser.compileParser derives skippedReasons map from built.skipped() and threads it to ParserGenerator.generate. The java25.peg grammar replaces split per-alternative whitespace with a folded choice wrapped in *, aligning lexer behavior with per-iteration absorption.
Shift operator inline expansion regression test suite peglib-core/src/test/java/org/pragmatica/peg/v6/generator/ShiftOperatorInlineExpansionTest.java	New comprehensive test class validates shift operators across field initializers, local initializers, shift-assignment contexts, generic type closers, and adjacent-token non-fusion cases. Includes a loud-guard regression ensuring parser generation rejects references to skipped lexer rules with incompatible negative lookaheads.
Trivia classification test suite enabling and unsplit %whitespace coverage peglib-core/src/test/java/org/pragmatica/peg/v6/lexer/TriviaClassificationTest.java	Re-enables previously disabled block-comment and mixed-trivia tests. Introduces "0.6.2 — unsplit %whitespace" section with paired GRAMMAR_UNSPLIT and GRAMMAR_SPLIT variants. New tests verify token-array parity across mixed trivia, doc variants, consecutive comments, assert no empty tokens, no DFA warnings, and input reconstruction round-trip.
Self-host diagnostic test suite re-enabling peglib-core/src/test/java/org/pragmatica/peg/v6/diagnostic/Java25SelfHostDiagTest.java	Removes @Disabled annotation from selfHostFixtureParsesCleanly and selfHostFixtureProducesShallowCST tests, activating both tests to validate the parser against the Java 25 self-host fixture.
Release notes, documentation, and version bump to 0.6.2 CHANGELOG.md, README.md, docs/ARCHITECTURE-0.6.0.md, docs/GRAMMAR-DSL.md, docs/HANDOVER.md, docs/VISITOR-TUTORIAL.md, peglib-core/pom.xml, peglib-formatter/pom.xml, peglib-incremental/pom.xml, peglib-maven-plugin/pom.xml, peglib-playground/pom.xml, peglib-runtime/pom.xml, pom.xml	CHANGELOG documents shift-operator and per-iteration %whitespace fixes, new test suite, and loud fromGrammar failures. README, documentation, and HANDOVER narrative are updated with 0.6.2 references and refined documentation of inline expansion and trivia classification. All pom.xml files and parent pom version are bumped from 0.6.1 to 0.6.2.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Poem

🐰 A shift-operator hops through the parse,
No longer stuck when generics close,
Per-iteration whitespace clasps,
Each trivia kind right where it goes—
0.6.2 bounds ahead with grace! 🌟

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch release-0.6.2

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}