Skip to content

Latest commit

 

History

History
34 lines (31 loc) · 1.7 KB

Invariants.md

File metadata and controls

34 lines (31 loc) · 1.7 KB

Invariants

This documentation was auto-generated using this parser to help dogfood the API. Please contribute fixes to tools/PrintInvariants.php and suggest API improvements.

We define and test both parser and lexer against a set of invariants (characteristics about the produced token set or tree that always hold true, no matter what the input). This set of invariants provides a consistent foundation that makes it easier to ensure the tree is "structurally sound", and confidently reason about the tree as we continue to build up our understanding.

Token Invariants

  • Sum of the lengths of all the tokens should be equivalent to the length of the document.
  • A token's Start is always >= FullStart.
  • A token's content exactly matches the range of the file its span specifies
  • FullText of each token matches Trivia plus Text
  • Concatenating FullText of each token returns the document
  • a token's FullText length is equivalent to Length - (Start - FullStart)
  • a token's Trivia length is equivalent to (Start - FullStart)
  • End-of-file token text should have zero length
  • Tokens array should always end with end of file token
  • Tokens array should contain exactly one EOF token
  • Token FullStart should begin immediately after previous token end
  • SkippedToken length should be greater than 0
  • MissingToken length should be equal to 0

Node Invariants

  • All invariants of Tokens
  • The tree length exactly matches the file length.
  • All Nodes have at least one child. $encode
  • Span of any Node is span of child nodes and tokens.
  • Parent of Node contains same child node.
  • each child has exactly one parent.
  • Every child is Node or Token type
  • Root node of tree has no parent.
  • root node of tree is never a child.