Error when inserting UTF-8 string to sqlite3 db with db.insert "ProgrammingError: You must not use 8-bit bytestrings" #264

Larpon opened this Issue Nov 13, 2013 · 0 comments


None yet
1 participant

Larpon commented Nov 13, 2013

Hi there,

Awesome little python webapp foundation this.
I've come across this error:

File "/home/test/modules/collection/", line 65, in scan_collection
    n = db.insert('collection', title='Test',hash=hash,file=filename)
File "/home/test/lib/web/", line 776, in insert
    self._db_execute(db_cursor, q1)
File "/home/test/lib/web/", line 589, in _db_execute
    out = cur.execute(query, params)

ProgrammingError: You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings.

Which is, as far as I can research, an error that shows when you try to insert an UTF-8 string into a sqlite3 db with the web db.insert() function - it puzzles me that no one have had this issue yet.

Now I've come across 2 solutions for this scenario:

  1. use the web.utils.safeunicode() function around the strings you want to put in your db
  2. patch by inserting this code on linie 1018:
if db.__name__ in ["sqlite3"]:
            db.register_adapter( str, lambda s:s.decode( 'utf-8' ) )

I don't know if it will work with other sqlite drivers but it fixes the issue.
(I will try to make a pull request on this)

I found the fix in an unrelated thread at:
Some old bug report at

Larpon pushed a commit to Larpon/webpy that referenced this issue Nov 13, 2013

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment