Consider adding ambiguity check for regexes

[matklad]

Hi! LALRPOP’s lexer has a very cool feature: if two tokens have regexes with non-empty intersection language, then an error is reported, unless there’s an explicit precedence annotation as well. This helps a lot with producing correct lexers: after implementing a TOML lexer with LALRPOP, I’ve found an ambiguity in the lexer I’ve implemented for IntelliJ Rust:
https://github.com/intellij-rust/intellij-rust/pull/2426/files

It would be cool if logos had this feature as well!

[matklad]

For the reference, here’s the relevant bit from LALRPOP:

https://github.com/lalrpop/lalrpop/blob/28533afc7fe0ba653d015ba587e0cda52b0840c8/lalrpop/src/lexer/dfa/mod.rs#L39

The precedence thing is what determines if ambiguity is an errror or not.

[maciejhirsz]

Logos currently errors if two patterns can resolve to the same token. Adding precedence markers to make it resolve those situations should be possible, will need to think about how to reflect it on the API surface without adding too much bloat to the attributes.