Debug toolbar crashes when using "latin-1" or non "utf-8" database #282

Closed
PaoloC68 opened this Issue May 20, 2012 · 2 comments

Projects

None yet

2 participants

@PaoloC68

I have my app (DJ 1.4) running nicely on a latin-1 database, this my connection:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql', # Add 'postgresql_psycopg2', 'postgresql', 'mysql', 'sqlite3' or 'oracle'.
        'NAME': 'wedirectory',                      # Or path to database file if using sqlite3.
        'USER': 'user',                      # Not used with sqlite3.
        'PASSWORD': 'password',                  # Not used with sqlite3.
        'HOST': 'localhost',                      # Set to empty string for localhost. Not used with sqlite3.
        'PORT': '3306',                      # Set to empty string for default. Not used with sqlite3.
        'OPTIONS': {
                    'charset': 'latin1',

                },
    }
}

It works all fine and especially even if I specify this in the settings:

DEFAULT_CHARSET = 'utf-8'

Everything is nicely shown in utf-8 and Django does perfectly his job but if I activate the:

'debug_toolbar.middleware.DebugToolbarMiddleware',

and try to update a record with the admin interface it throws errors:

DjangoUnicodeDecodeError: 'utf8' codec can't decode byte 0xe7 in position 17: invalid continuation byte. You passed in 'Taverna Pub - Ter\xe7a de Bambas - Natal' (<type 'str'>)

and even worse if you set django to work with latin-1

DEFAULT_CHARSET = 'latin-1'

it crashes badly:

django.utils.encoding.DjangoUnicodeDecodeError
DjangoUnicodeDecodeError: 'utf8' codec can't decode byte 0xe7 in position 236: invalid continuation byte. You passed in  '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">\n<html xmlns="http://www.w3.org/1999/xhtml" lang="pt-br" xml:lang="pt-br" 

So seems that DDT is not compatible with apps they don't use purely UTF-8.

:( So sad!

@aaugustin

The second problem (DEFAULT_CHARSET != 'utf-8') was reported in #15 and is now fixed.

I'll keep this ticket open for the first one.

@aaugustin

I cannot reproduce this problem. I've done a lot of work on encoding issues over the last month, and I may have fixed it.

I'm seeing some badly decoded data in the SQL panel as expected:

SELECT (1) AS `a` FROM `auth_group` WHERE `auth_group`.`name` = '���' LIMIT 1

Clearly the DDT is seeing latin-1 encoded data and dealing with it properly.

If you still see it with the master branch, please reopen the ticket and provide a full traceback.

@aaugustin aaugustin closed this Nov 17, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment