You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
As shown by the test program below, Perl_utf8n_to_uvuni will decode
characters that Perl_uvuni_to_utf8_flags considers illegal. The problem
characters are U+FDD0 through U+FDEF, U+FFFE, U+xFFFE for 1 <= x <= 10,
and U+xFFFF for 1 <= x <= 10. The two functions must agree as to what
is an illegal character or programs that handle untrusted input will
have insufficient control over what perl warnings get thrown.
use Encode;
use strict;
use warnings;
sub trydecode {
my ($utf8) = (@_);
my $text = Encode::decode('UTF-8', $utf8, 0);
The behavior is now changed so that all of these silently turn into the
Unicode replacement character, U+FFFD. Both routines now know about the
same 66 Unicode non-character code points.
If you want a warning, a ticket should be written against Encode in CPAN
Activity
p5pRT commentedon Mar 13, 2006
From jgmyers@proofpoint.com
Created by jgmyers@pong.us.proofpoint.com
As shown by the test program below, Perl_utf8n_to_uvuni will decode
characters that Perl_uvuni_to_utf8_flags considers illegal. The problem
characters are U+FDD0 through U+FDEF, U+FFFE, U+xFFFE for 1 <= x <= 10,
and U+xFFFF for 1 <= x <= 10. The two functions must agree as to what
is an illegal character or programs that handle untrusted input will
have insufficient control over what perl warnings get thrown.
use Encode;
use strict;
use warnings;
sub trydecode {
my ($utf8) = (@_);
my $text = Encode::decode('UTF-8', $utf8, 0);
printf "%x\n", ord(substr($text, 3, 1));
$text =~ /\b(?:https?|ftp)/o;
}
trydecode("aaa\xef\xbf\xbebbb"); #fffe
trydecode("aaa\xef\xbf\xbfbbb"); #ffff
trydecode("aaa\xef\xb7\x90bbb"); #fdd0
trydecode("aaa\xf0\x9f\xbf\xbebbb"); #1fffe
trydecode("aaa\xf0\x9f\xbf\xbfbbb"); #1ffff
Perl Info
p5pRT commentedon Jan 10, 2011
From @khwilliamson
The behavior is now changed so that all of these silently turn into the
Unicode replacement character, U+FFFD. Both routines now know about the
same 66 Unicode non-character code points.
If you want a warning, a ticket should be written against Encode in CPAN
--Karl Williamson
p5pRT commentedon Jan 10, 2011
The RT System itself - Status changed from 'new' to 'open'
p5pRT commentedon Jan 10, 2011
@khwilliamson - Status changed from 'open' to 'resolved'