TypeError: String or Integer object expected for key, unicode found

[sqlalchemy-bot]

Migrated issue, originally created by Antoine Bertin (diaoul)

  File "/home/antoine/Documents/Workspace/subliminal/subliminal/services/bierdopje.py", line 56, in query
    request_id = self.get_show_id(series.lower())
  File "/home/antoine/.virtualenvs/subliminal/local/lib/python2.7/site-packages/dogpile/cache/region.py", line 500, in decorate
    return self.get_or_create(key, creator, expiration_time)
  File "/home/antoine/.virtualenvs/subliminal/local/lib/python2.7/site-packages/dogpile/cache/region.py", line 365, in get_or_create
    expiration_time) as value:
  File "/home/antoine/.virtualenvs/subliminal/local/lib/python2.7/site-packages/dogpile/core/dogpile.py", line 142, in __enter__
    return self._enter()
  File "/home/antoine/.virtualenvs/subliminal/local/lib/python2.7/site-packages/dogpile/core/dogpile.py", line 83, in _enter
    value = value_fn()
  File "/home/antoine/.virtualenvs/subliminal/local/lib/python2.7/site-packages/dogpile/cache/region.py", line 346, in get_value
    value = self.backend.get(key)
  File "/home/antoine/.virtualenvs/subliminal/local/lib/python2.7/site-packages/dogpile/cache/backends/file.py", line 168, in get
    value = dbm.get(key, NO_VALUE)
  File "/home/antoine/.virtualenvs/subliminal/lib/python2.7/_abcoll.py", line 342, in get
    return self[key]
  File "/usr/lib/python2.7/bsddb/__init__.py", line 270, in __getitem__
    return _DeadlockWrap(lambda: self.db[key])  # self.db[key]
  File "/usr/lib/python2.7/bsddb/dbutils.py", line 68, in DeadlockWrap
    return function(*_args, **_kwargs)
  File "/usr/lib/python2.7/bsddb/__init__.py", line 270, in <lambda>
    return _DeadlockWrap(lambda: self.db[key])  # self.db[key]
TypeError: String or Integer object expected for key, unicode found

Might be the first time I encounter a module able to work with bytestrings but not unicode... This is usually the other way around.

Could you fix this by encoding/decoding with utf-8 by default?
Don't know if this is the same for other backends but not being able to use unicode is a real problem IMO.

[sqlalchemy-bot]

Antoine Bertin (diaoul) wrote:

I get it, function_key_generator in dogpile.cache.util returns unicode even though I give my function a pure bytestring.
http://dogpilecache.readthedocs.org/en/latest/api.html#dogpile.cache.util.function_key_generator

Also, that's not unittested.

[sqlalchemy-bot]

Antoine Bertin (diaoul) wrote:

Quick fix:

from dogpile.cache import make_region
from dogpile.cache.util import function_key_generator

def my_key_generator(namespace, fn):
    def generate_key(*args):
        key = function_key_generator(namespace, fn)(*args)
        if isinstance(key, unicode):
            key = key.encode('utf-8')
        return key
    return generate_key

region = make_region(function_key_generator=my_key_generator)

Not ideal

[sqlalchemy-bot]

Michael Bayer (zzzeek) wrote:

yeah this is a dupe of #6. I'm still sticking with "use key_mangler" as the official way to handle this, as that's what it's there for. I don't like adding a big assumption of "unicode" + "happens to be utf-8" + "all backends need it".

But I do need to figure out a way to make this very clear because I understand how people are missing this.

[sqlalchemy-bot]

Michael Bayer (zzzeek) wrote:

Duplicate of #6.

[sqlalchemy-bot]

Changes by Michael Bayer (zzzeek):

• added labels: duplicate
• changed status to closed

[sqlalchemy-bot]

Michael Bayer (zzzeek) wrote:

Duplicate of #6.

[sqlalchemy-bot]

Antoine Bertin (diaoul) wrote:

I think you missed it, I passed a bytesting here, not unicode, yet I have an error about unicode.
I'll try to make an example.

[sqlalchemy-bot]

Antoine Bertin (diaoul) wrote:

There you go:

from dogpile.cache import make_region

region = make_region()
region.configure('dogpile.cache.dbm', arguments={'filename': 'test.dbm'})

@region.cache_on_arguments()
def test(name):
    return 1


if __name__ == '__main__':
    print test('foo')

[sqlalchemy-bot]

Michael Bayer (zzzeek) wrote:

this is something else happening.

[sqlalchemy-bot]

Changes by Michael Bayer (zzzeek):

• removed labels: duplicate
• changed status to reopened

[sqlalchemy-bot]

Michael Bayer (zzzeek) wrote:

found #15 on the way there...

[sqlalchemy-bot]

Michael Bayer (zzzeek) wrote:

fixed the inadvertent coercion to unicode in c9c29f4.

[sqlalchemy-bot]

Changes by Michael Bayer (zzzeek):

• changed status to closed