Skip to content

v0.7.0

Choose a tag to compare

@dsisco11 dsisco11 released this 03 Jan 01:44
· 23 commits to master since this release
bb3553c

Release: v1.x.x - Standardized Serialization & Trivia Preservation

🎯 Highlights

This release introduces a unified text serialization API and fixes a critical bug affecting whitespace preservation in nested code structures.


✨ New Features

Unified ITextSerializable Interface

All parsed structures now implement a consistent interface for serializing back to text:

// High-performance buffer-based serialization
token.WriteTo(bufferWriter);

// Get text as string
string text = token.ToText();

// Write to StringBuilder or TextWriter
token.WriteTo(stringBuilder);
token.WriteTo(streamWriter);

// Zero-allocation span-based writing
if (token.TryWriteTo(stackBuffer, out int written))
{
    // Use stackBuffer[..written]
}

// Pre-allocate with known length
int length = token.TextLength;

Implemented by: Token, SimpleToken, GreenNode, RedNode, SyntaxTree


🐛 Bug Fixes

Fixed: Whitespace Lost in Nested Blocks

Issue: When parsing code with nested blocks (e.g., nested if statements, loops), indentation before closing braces was being lost during serialization.

Before:

if (a) {
    if (b) {
        x = 1;
    }
} else {      // ❌ Indentation lost: "} else {" instead of "    } else {"
    y = 0;
}

After: All whitespace and trivia is now correctly preserved through parse → edit → serialize cycles.


⚠️ Breaking Changes

ToString() Behavior Changed

ToString() now returns a debug representation instead of serialized text:

// Old behavior (removed)
token.ToString()  // returned "actual token content"

// New behavior
token.ToString()  // returns "IdentToken@42" (debug info)
token.ToText()    // returns "actual token content" ✅

Migration: Replace all .ToString() calls with .ToText() when you need the serialized content.

Deprecated Methods

Deprecated Replacement
SyntaxTree.Serialize() SyntaxTree.ToText()
SyntaxTree.ToFullString() SyntaxTree.ToText()

GreenBlock Constructor (Internal API)

If you're using the internal GreenBlock constructor directly, use the new factory method:

// Old (no longer works)
new GreenBlock('{', children, leadingTrivia, trailingTrivia)

// New
GreenBlock.Create('{', children, openerLeadingTrivia, openerTrailingTrivia, 
                  closerLeadingTrivia, closerTrailingTrivia)

📊 Testing

  • 70+ new tests for block serialization roundtrips
  • Coverage for deeply nested structures (up to 100 levels)
  • Real-world GLSL shader parsing scenarios
  • Mixed line endings and whitespace preservation
  • Undo/redo correctly restores exact original text

🔧 Internal Improvements

  • GreenBlock redesigned with explicit opener/closer GreenLeaf nodes for proper trivia attachment
  • GreenNodeCache.CreateDelimiter() for efficient delimiter node creation
  • All serialization now routes through IBufferWriter<char> for consistent, high-performance output