Document Lexer Rules

[valarauca]

Depending on the lexiconal order the EBNF rules, what is and is not a valid parse changes.

Now to a degree this makes sense, but most implementations resolve this ahead of time.

But for example if I have an EBNF like

(. changed to ; for github syntax highlighting)

num = "0" | "1" | "2" | "3" | "4" | "5" | "6" | "7" | "8" | "9" ;
Version = { num } [ "." { num } [ "." { num } ] ] ;

letter = "a" | "b" | "c" | "d" | "e" | "f" | "g" | "h" | "i" | "j" | "k" | "l" | "m" | "n" | "o" | "p" | "q" | "r" | "s" | "t" | "u" | "v" | "w" | "x" | "y" | "z" ;
package_name = letter | num | "_" | "-" | "." ;
PackageID = letter { package_name } ;

This will correctly parse for @Version and @PackageID

But if I add the rule:

NameID = ID ":" ID ":"  Version ;

Anything tagged as @PackageID breaks because it doesn't contain a : character.

[alecthomas]

Can you provide an example?

[alecthomas]

Also, have you tried latest master? I just merged the lookahead branch which should since this.

[alecthomas]

Sorry, was on mobile. I assume you're talking about the EBNF lexer. Can you provide the full lexer you're using that breaks, and an example grammar that fails? (your example above is missing the definition for ID).

[valarauca]

Just me being a noob very sorry