Unicode error

[Christophe31]

Hi, I followed your documentation's suggestion to use django sendfile.

I'm French and as English persons we have some word with accentuation like a café.

It's rather common for us to have this kind of letters in file names, but here a stacktrace:

[my app things]
  File "/home/gunicorn/prod/ama/ama_prod/ama_app/models/site.py", line 186, in get_document
    return sendfile(request, self.file.path)

  File "/home/gunicorn/prod/ama/venv/lib/python2.6/site-packages/sendfile/__init__.py", line 59, in sendfile
    response = _sendfile(request, filename, mimetype=mimetype)

  File "/home/gunicorn/prod/ama/venv/lib/python2.6/site-packages/sendfile/backends/nginx.py", line 7, in sendfile
    response['X-Accel-Redirect'] = _convert_file_to_url(filename)

  File "/home/gunicorn/prod/ama/venv/lib/python2.6/site-packages/django/http/__init__.py", line 612, in __setitem__
    header, value = self._convert_to_ascii(header, value)

  File "/home/gunicorn/prod/ama/venv/lib/python2.6/site-packages/django/http/__init__.py", line 601, in _convert_to_ascii
    value = value.encode('us-ascii')

UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 40: ordinal not in range(128), HTTP response headers must be in US-ASCII format

note: my file name is péter_là_gueule.txt

Using django 1.5 and gunicorn with UTF-8 encoded filesystem.

[johnsensible]

Hmmm. Well it appears that the trouble is that Django is trying to convert the filename to ascii.

Looks like I probably just need to url encode the file name, before setting in the the header:

http://stackoverflow.com/questions/2732268/django-unicode-filenames-with-ascii-headers

Will hopefully get round to this later - unless you fancy submitting a pull request?

[Christophe31]

Ok, I'm trying it, if it works, I submit a pull request.

[Christophe31]

Hoped it was that simple.

  File "/home/gunicorn/prod/ama/venv/lib/python2.6/site-packages/sendfile/__init__.py", line 59, in sendfile
    response = _sendfile(request, filename.encode('utf-8'), mimetype=mimetype)

  File "/home/gunicorn/prod/ama/venv/lib/python2.6/site-packages/sendfile/backends/nginx.py", line 7, in sendfile
    response['X-Accel-Redirect'] = _convert_file_to_url(filename)

  File "/home/gunicorn/prod/ama/venv/lib/python2.6/site-packages/sendfile/backends/_internalredirect.py", line 13, in _convert_file_to_url
    return u'/'.join(url)

UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1: ordinal not in range(128)

[Christophe31]

(the worse part is that those accentuated characters used to be in extended ASCII)

[johnsensible]

Ok do you want to give that a go? I've added some tests to check for unicode characters and encode them properly to master.

[Christophe31]

It's great, I tested it in my use-case, it works.

Thanks a lot.

[johnsensible]

Ok good. It's also on PyPI now, so should be pip installable.