Suggest providing byte position spans

[FranklinChen]

I noticed that generated yytext is a freshly allocated String copying from the original '&'a str passed to the lexer, and also the internal token positions zz_start_read and zz_marked_pos are char indices. I suggest an additional interface in which byte offsets are also stored, by means of using https://doc.rust-lang.org/std/primitive.str.html#method.char_indices to keep track of byte offsets, and then it would be possible to provide a method that returns a Range https://doc.rust-lang.org/std/ops/struct.Range.html instead of yytext, and then the client could use that along with the original &'a str to read a substring if desired, all without any new allocation (except for a Range pair of byte offsets).

[suma]

Thank you for suggestion and creating this issue.
I released v0.7.0 that contains new function yytextpos which returns Range type.

[suma]

Well, I implemented to return zz_start_read/zz_marked_pos directly. If you want byte offsets, I re-think lexer implementation and yy??? interface again in one of these days.

[suma]

I work at master...suma:use-byte-offset . Almost functions are done, but yypushback is only buggy now...

[suma]

I've released v0.8.0 which uses char_indices in lexer, of course we can get byte position of accepted string.