Skip to content
This repository

Fixed unhandled UnicodeDecodeError in utils.sqlparse.lexer #372

Closed
wants to merge 2 commits into from

3 participants

Ivo van Doesburg Aymeric Augustin Vladislav
Ivo van Doesburg

This fixes an UnicodeDecodeError exception when the query contains a value for a mysql BINARY field. As I don't care about the value in de debug toolbar context I've replaced the text with u'[UnicodeDecodeError]'

Vladislav midiotthimble commented on the diff April 25, 2013
debug_toolbar/utils/sqlparse/lexer.py
@@ -240,7 +240,10 @@ def get_tokens(self, text, unfiltered=False):
240 240
                 enc = chardet.detect(text)
241 241
                 text = text.decode(enc['encoding'])
242 242
             else:
243  
-                text = text.decode(self.encoding)
  243
+                try:
  244
+                    text = text.decode(self.encoding)
1

smart_unicode better.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Ivo van Doesburg

I just found out that this issue has been fixed in andialbrecht/sqlparse@edcd02f

Probably better to do it that way

Aymeric Augustin
Owner

The Python 3 support branch (#412) deprecates the vendored copy of sqlparse.

Since the problem was fixed in sqlparse itself, we can close this ticket.

Aymeric Augustin aaugustin closed this October 17, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Showing 2 unique commits by 1 author.

Apr 05, 2013
Fixed unhandled UnicodeDecodeError in utils.sqlparse.lexer 9528520
Apr 26, 2013
Unicode-escape for binary data (sqlparse/lexer.py) 244dabd
This page is out of date. Refresh to see the latest.

Showing 1 changed file with 4 additions and 1 deletion. Show diff stats Hide diff stats

  1. 5  debug_toolbar/utils/sqlparse/lexer.py
5  debug_toolbar/utils/sqlparse/lexer.py
@@ -240,7 +240,10 @@ def get_tokens(self, text, unfiltered=False):
240 240
                 enc = chardet.detect(text)
241 241
                 text = text.decode(enc['encoding'])
242 242
             else:
243  
-                text = text.decode(self.encoding)
  243
+                try:
  244
+                    text = text.decode(self.encoding)
  245
+                except UnicodeDecodeError:
  246
+                    text = text.decode('unicode-escape')
244 247
         if self.stripall:
245 248
             text = text.strip()
246 249
         elif self.stripnl:
Commit_comment_tip

Tip: You can add notes to lines in a file. Hover to the left of a line to make a note

Something went wrong with that request. Please try again.