Skip to content
Perl6 Supply which takes a supply of bytes as input and returns a supply of unicode characters
Other Perl
  1. Other 77.5%
  2. Perl 22.5%
Branch: master
Clone or download

Latest commit

Fetching latest commit…
Cannot retrieve the latest commit at this time.

Files

Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
lib/Unicode
t
.gitignore
.travis.yml
LICENSE
META6.json
README.md

README.md

Unicode::UTF8-Parser

Rakudo's built-in UTF8 parser will wait for a possible combining character before getc() returns. Using this module, you can read bytes from $*IN and use this module to get unicode characters

use Unicode::UTF8-Parser;

my $stdin = supply { while (my $b = $*IN.read(1)[0]).defined { emit($b) } };
my $utf8 = parse-utf8-bytes($stdin);

$utf8.tap({ say "got $_" });

The module exports the sub parse-utf8-bytes, which take a supply as input and returns a new supply.

If there are Non-UTF8 bytes in the stream, they will be emitted as Int. You have to implement handling of these yourself ($val ~~ Int).

You can’t perform that action at this time.