Skip to content

rust: define Token, Span, and ParseError types (parser ABI freeze) #552

@DecisionNerd

Description

@DecisionNerd

Context

The parser migration strategy requires differential testing between the existing Python LALR(1) parser and the new LALRPOP Rust parser. To make this possible both parsers must emit the same token contract. This issue freezes that contract before either parser is written.

Deliverables

  • `Token` enum in `gf-ast` covering all openCypher token classes (keywords, literals, punctuation, identifiers, parameters)
  • `Span { start: u32, end: u32 }` struct — byte offsets into the source string
  • `ParseError { kind: ParseErrorKind, span: Span, message: String }` struct with typed error kinds (unexpected token, unknown identifier, unterminated string, etc.)
  • All three types are `Clone + Debug + PartialEq` and serialisable via Serde
  • Unit tests asserting span arithmetic and error formatting
  • Document the token contract in a doc comment on the `Token` enum

Exit criteria

The token types are stable enough that the LALRPOP parser (next issue) and the differential test harness (later issue) can both depend on them without changes.

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestparserChanges to Cypher parser

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions