UnicodeEncodeError when script is in a folder with non-ascii characters in the name #149

Closed
jjunho opened this Issue Dec 9, 2010 · 6 comments

Comments

Projects
None yet
7 participants

jjunho commented Dec 9, 2010

Very very nice work done in Flask! Really liked it!
However, I was trying to run my Flask script under a subfolder of my Desktop folder in Ubuntu. Since my system is set up for Portuguese (I'm Brazilian), the default name for the folder is "Área de Trabalho". I was getting UnicodeEncodeError every time I tried to run the example in "11.7 Uploading Files". (At least it was really good to teach me how to use the Flask debugger!) I went around, examining where the error was coming from, and, finally, got to "File "/usr/local/lib/python2.6/dist-packages/Flask-0.6-py2.6.egg/flask/helpers.py", line 350, in send_file", when the function adler32 uses the unicode string. Then, I just tried a quick "from zlib import adler32; adler32(u'Á')" to realize that zlib doesn't allow unicode strings, returning a "UnicodeEncodeError: 'ascii' codec can't encode character u'\xc1' in position 0: ordinal not in range(128)".
Of course I can move my script to another folder, but it would be nice not having to think about that in the future, especially when it is said that "Flask like Jinja2 and Werkzeug is totally Unicode based when it comes to text".
Sorry if I don't make much sense... it's almost 3:00am around here...

Congrats for the great job on Flask!

jjunho

Owner

mitsuhiko commented Dec 9, 2010

Heh. I really should have included an exception for "except for filesystem access". The reason for this being that it's close to impossible to support unicode file access on POSIX systems. But it's definitively something that should be improved, just that it's damn hard to get it right.

Contributor

dgerber commented Mar 22, 2012

What about wrapping path operations like this?

try:
    return os.path.join(directory, filename)
except UnicodeDecodeError:
    encoding = sys.getfilesystemencoding()
    d, f = [x if isinstance(x, unicode) else unicode(x, encoding) for x in (directory, filename)]
    return os.path.join(d, f)

sprhawk commented Dec 6, 2012

I encounter the problem and fixed by:

create database xxx default character set utf8 default collate utf8_gerneral_ci

in schema.sql:

set names 'utf8' colate 'utf8_general_ci'

in sqlalchemy connection uri:

mysql://test:test@localhost/test?charset=utf8

then it works well in Jinja2

booo commented Jan 12, 2013

It would be extremely useful if we fix this problem or add a comment to the documentation that tells the user not to use a path that contains characters other the ascii charset.

If you point me in the right direction I can add the comment to the documentation.

Member

jeffwidman commented Jun 3, 2016

The original bug was reported against Flask 0.6.

Is this issue still relevant?

Contributor

antsar commented Jun 3, 2016

Tried this on 0.7 thru master and could not reproduce. Flask examples run fine even when the path contains non-ascii chars.

@jeffwidman jeffwidman closed this Jun 3, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment