Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Fixed CompilerFilter to always output Unicode strings #312

Merged
merged 1 commit into from Apr 3, 2013

Conversation

Projects
None yet
2 participants
Contributor

anttihirvonen commented Sep 12, 2012

This fixes #311.

@jezdez jezdez added a commit that referenced this pull request Apr 3, 2013

@jezdez jezdez Merge pull request #312 from anttihirvonen/precompiler_unicode_output…
…_fix

Fixed CompilerFilter to always output Unicode strings
8242424

@jezdez jezdez merged commit 8242424 into django-compressor:develop Apr 3, 2013

1 check passed

default The Travis build passed
Details

@gldnspud gldnspud added a commit to 11craft/django_compressor that referenced this pull request Apr 4, 2013

@gldnspud gldnspud Encode compiled content when writing it w/o compression
As of #312, the output of compiled content is always unicode.

With ``COMPRESS_ENABLED=True``,
that unicode was piped directly to compression
without writing the intermediate content.
The byte encoding of the compressed content
is handled correctly in this case.

However, with ``COMPRESS_ENABLED=False``,
the unicode content was sent directly to a
``django.core.files.base.ContentFile`` instance,
and somewhere in there
it turned that content into 4-byte-wide characters
resembling UTF-32 encoded content.

This encodes that content using UTF-8
before handing it off to the ``ContentFile``.

This may present an issue in a corner case
where something other than UTF-8 encoding is desired.
a20c905

@liamcurry liamcurry pushed a commit to liamcurry/django_compressor that referenced this pull request Apr 30, 2013

@gldnspud gldnspud + Liam Curry Encode compiled content when writing it w/o compression
As of #312, the output of compiled content is always unicode.

With ``COMPRESS_ENABLED=True``,
that unicode was piped directly to compression
without writing the intermediate content.
The byte encoding of the compressed content
is handled correctly in this case.

However, with ``COMPRESS_ENABLED=False``,
the unicode content was sent directly to a
``django.core.files.base.ContentFile`` instance,
and somewhere in there
it turned that content into 4-byte-wide characters
resembling UTF-32 encoded content.

This encodes that content using UTF-8
before handing it off to the ``ContentFile``.

This may present an issue in a corner case
where something other than UTF-8 encoding is desired.

Signed-off-by: Liam Curry <liam@curry.name>
4976392

@gldnspud gldnspud added a commit to 11craft/django_compressor that referenced this pull request May 1, 2013

@gldnspud gldnspud Only encode to ``utf8`` if content is unicode. - #312
Before this change,
when the ``compress`` management command is used
in conjunction with the ``COMPRESS_OFFLINE = True`` setting,
an error would occur if there were non-ASCII characters:

    Compressing... Error: An error occured during rendering /path/to/template/filename.html: 'ascii' codec can't decode byte 0x.. in position ####: ordinal not in range(128)
c83addf
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment