Serializers and UTF-8 #882

Open
rizen opened this Issue Jan 8, 2013 · 1 comment

Comments

Projects
None yet
2 participants
Contributor

rizen commented Jan 8, 2013

Serializers are broken when it comes to UTF-8 encoding. Let me give you an example. If you turn on utf8 in the JSON engine then this string:

UTF test ûçæ ΦѺ ݰݮݭ ୩୪୫୬୯

becomes:

UTF test ûçæ ΦѺ ݰݮݭ ୩୪୫

This is due to being double encoded. However, if you don't enable utf8 in the JSON engine then you get wide character errors. Dancer already has a way of dealing with this by setting the "encoded" flag on the response object. But the serializers don't have a way of setting that because they don't have a reference to the response object.

If you add this to line 64 of Serializer.pm it fixes the problem for JSON:

use Dancer::Config 'setting';
$response->{encoded} = 1 if engine->{name} eq 'JSON' && setting('engines')->{JSON}{utf8};

But this is of course a terrible fix. A better way would be to add a flag to the serializers so the Serializer.pm could ask them whether or not they encoded the content.

For anybody currently experiencing this problem a quick and dirty work around is to add this hook to your code:

hook(
   after_serializer => sub {
       my $response = shift;
       $response->{encoded} = 1;
   }
);

@nichtich nichtich referenced this issue in PerlDancer/Dancer2 Jan 15, 2014

Closed

Support all JSON configuration options #520

Hello,

has the fix been tested ? right now I have an x86 linux not showing the issue and an arm one showing it, and adding the hook doesn't help...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment