source of template string determines output encoding under `use utf8;` #54

Closed
wchristian opened this Issue Dec 24, 2011 · 1 comment

Comments

Projects
None yet
2 participants
Contributor

wchristian commented Dec 24, 2011

I have uploaded a minimal test case for this issue here:

http://dl.dropbox.com/u/10190786/encoding.zip

What happens for me is that test_result_works is written as a file with proper unicode content; but writing test_result_broken causes the script to die with an error about wide characters.

The difference between those is that the working file is written with a template defined within the perl code itself; while the broken file template comes from the DATA section. Earlier testing indicates that in the first case, Text::Xslate returns bytes and in the second case character. use utf8 is in effect for both of those, so the difference in behavior is unexpected.

Member

punytan commented Dec 24, 2011

How about this verbose version?
You should see FLAGS carefully.
See also perlunifaq

use strictures;

package CPANRSS;

use utf8;
use File::Slurp qw' write_file read_file ';
use Data::Section::Simple 'get_data_section';
use Text::Xslate;
use Devel::Peek;
use Encode;
$| = 1;

save_feed();

sub save_feed {
    my $name = read_file 'test_utf8';
    show("name", $name);
    $name = decode_utf8 $name;
    show("name", $name);

    my $works = Text::Xslate->new->render_string( 'wagh: <: $name :>', { name => $name } );
    show("works", $works);

    $works = encode_utf8($works);
    show("works", $works);

    write_file 'test_result_works', { binmode => ':raw' }, $works;

    my $template = get_data_section( 'meep' );
    show("template", $template);

    my $broken = Text::Xslate->new->render_string( $template, { name => $name } );
    show("broken", $broken);

    $broken = encode_utf8($broken);
    show("broken", $broken);

    write_file 'test_result_broken', { binmode => ':raw' }, $broken;
}

sub show {
    print "\$$_[0]: ";
    Dump $_[1];
}

1;

__DATA__

@@ meep
wagh: <: $name :>

@wchristian wchristian closed this Dec 11, 2013

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment