UTF-8 and text #11

Closed
elliottt opened this Issue Jun 7, 2012 · 2 comments

2 participants

@elliottt

As alex-2.x.x worked off of characters, interesting encodings were supported automatically, by nature of Char being able to describe them. As libraries like text provide a way of decoding from multiple input formats, and interpreting them as a buffer of Chars, alex was more or less able to handle those formats.

With the change to expecting bytes as input in alex-3.x.x, using libraries like text means first decoding the original format, then re-encoding each character as a list of bytes that can then be given, one at a time, to alex. Is this the best way to use a library like text with alex, or is there a better way that isn't covered in the documentation?

@simonmar
Owner
@elliottt

Thanks, I'll just switch to using a ByteString as the input to the lexer :)

@elliottt elliottt closed this Jun 8, 2012
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment