Skip to content

Commit

Permalink
Made :utf8 an actual layer
Browse files Browse the repository at this point in the history
It will check the input for validity, by default strict validity though less strict forms are provided. This also means PerlIO::get_layers doesn't return a "utf8" pseudo-layer anymore, which can break some code making that assumption.
  • Loading branch information
Leont committed May 2, 2021
1 parent e45d824 commit 3485f29
Show file tree
Hide file tree
Showing 15 changed files with 486 additions and 94 deletions.
5 changes: 4 additions & 1 deletion cpan/CPAN-Meta-YAML/t/11_read_string.t
Expand Up @@ -40,7 +40,10 @@ subtest 'invalid UTF-8' => sub {
# get invalid UTF-8 by reading Latin-1 with lax :utf8 layer
my $string = do {
local $SIG{__WARN__} = sub {};
slurp( test_data_file('latin1.yml'), ":utf8" );
my $ret = slurp( test_data_file('latin1.yml'), ":raw" );
require Encode;
Encode::_utf8_on($ret);
$ret;
};
my $obj = eval { CPAN::Meta::YAML->read_string($string); };
is( $obj, undef, "read_string should return undef" );
Expand Down
21 changes: 10 additions & 11 deletions lib/PerlIO.pm
@@ -1,6 +1,6 @@
package PerlIO;

our $VERSION = '1.11';
our $VERSION = '1.12';

# Map layer name to package that defines it
our %alias;
Expand Down Expand Up @@ -173,6 +173,8 @@ instead produce UTF-EBCDIC on EBCDIC systems. The C<:encoding(UTF-8)>
layer (hyphen is significant) is preferred as it will ensure translation
between valid UTF-8 bytes and valid Unicode characters.
Note that before perl 5.36, this layer did not validate byte sequences.
=item :bytes
This is the inverse of the C<:utf8> pseudo-layer. It turns off the flag
Expand Down Expand Up @@ -220,10 +222,9 @@ but then enable UTF-8 translation.
=item :pop
A pseudo-layer that removes the top-most layer. Gives Perl code a
way to manipulate the layer stack. Note that C<:pop> only works on
real layers and will not undo the effects of pseudo-layers or flags
like C<:utf8>. An example of a possible use might be:
A pseudo layer that removes the top-most layer. Gives perl code
a way to manipulate the layer stack. An example of a possible use
might be:
open(my $fh,...) or die "open failed: $!";
...
Expand Down Expand Up @@ -254,9 +255,8 @@ Some custom layers come with the Perl distribution.
=item :encoding
Use C<:encoding(ENCODING)> to transparently do character set and encoding
transformations, for example from Shift-JIS to Unicode. Note that an
C<:encoding> also enables C<:utf8>. See L<PerlIO::encoding> for more
information.
transformations, for example from Shift-JIS to Unicode. See
L<PerlIO::encoding> for more information.
=item :mmap
Expand Down Expand Up @@ -372,9 +372,8 @@ You are supposed to use open() and binmode() to manipulate the stack.
B<Implementation details follow, please close your eyes.>
The arguments to layers are by default returned in parentheses after
the name of the layer, and certain layers (like C<:utf8>) are not real
layers but instead flags on real layers; to get all of these returned
separately, use the optional C<details> argument:
the name of the layer; to get all of these returned separately, use the
optional C<details> argument:
my @layer_and_args_and_flags = PerlIO::get_layers($fh, details => 1);
Expand Down

0 comments on commit 3485f29

Please sign in to comment.