Unicode support

[Yamakaky]

Does your crate support Unicode character classes and normalization?

[Marwes]

No, you will have to look into some other crate for that. You can then use satisfy(|token| predicate(token)).expected("Expected value") to create a custom parser for whatever predicate you want (digit, letter etc are all implemented like this)

[jan-hudec]

For classes only, yes. For normalisation, no, one would have to write a new parser using the unicode-normalization crate, because a token here means a single codepoint (char), but normalisation needs to see multiple-codepoint sequences.

In fact I tried—and failed (#85)—at similar thing with graphemes and unicode-segmentation yesterday.

And then it depends on how generic you'd need the parser to be. unicode-normalization works on &str, so if you have that, you can use it directly, but if you have some other kind of stream, you have to collect and try normalising.

[Marwes]

To get regex parsers working I added FullRangeStream which lets one get retrieve a view into the entire parsers input.
Since unicode parsers require &str then this can be used to write parsers generic over the input stream (translating the example in #85 (comment).

pub fn grapheme<'a, I>(input: I) -> ParseResult<&'a str, I>
    where I: FullRangeStream<Range = &'a str>
{
    let mut iter = input.range().graphemes();
    match iter.next() {
        Some(g) => {
            let len = input.range().len() - iter.as_str().len();
            take(len).parse_stream(input)
        }
        None => Err(ParseError::end_of_input()),
    }
}