How do whitespaces get handled?

[Binero]

I am puzzled by how the library handles whitespace. I have the following syntax:

pub Identifier: String = <ident:r"[\_A-z][\_A-z0-9]*"> => String::from(ident);

I have a unit test where it tests the syntax for " Word". It passes. Why is this? It should fail.

[j16r]

I'm having a little trouble here also, I'm trying to create a grammar where newlines delimit statements, but there seems to be some implicit consumption of whitespace, my syntax only works with a delimiter other than "\n".

[nikomatsakis]

Yes, all of the lexers that LALRPOP generate assume whitespace is insignificant. Lifting this is the subject of https://github.com/nikomatsakis/lalrpop/issues/14. Probably one of the biggest remaining obstacles to 1.0 -- I just can't decide how I want it to work, among other things. :(

The only workaround right now is to hand-write a lexer, or else use a tool like https://github.com/LeoTestard/rustlex -- and indeed I think one short-term "fix" for #14 might just be more seamless "rustlex" integration (though I still want something more long-term).

[nikomatsakis]

Going to close as a dup of #14. Thanks all.