dynamodb2.table.Table.batch_write promotes str to unicode incorrectly. #1889

jschachter opened this Issue Dec 3, 2013 · 2 comments

3 participants


it appears that you can't use batch_write to load binary data.

r = { 'word': 'yes'}
r['s1'] = umsgpack.packb( { 's1': True } )

print type(r['s1'])
batch.put_item( data=r )

{'s1': '\x82\xc4\x01s\xc3\xc4\x01t\xc3', 'word': 'yes'}

Traceback (most recent call last):
File "test-dynamo", line 39, in
batch.put_item( data=r )
File "/usr/local/lib/python2.7/dist-packages/boto/dynamodb2/table.py", line 1083, in exit
File "/usr/local/lib/python2.7/dist-packages/boto/dynamodb2/table.py", line 1126, in flush
File "/usr/local/lib/python2.7/dist-packages/boto/dynamodb2/layer1.py", line 266, in batch_write_item
File "/usr/lib/python2.7/json/init.py", line 231, in dumps
return _default_encoder.encode(obj)
File "/usr/lib/python2.7/json/encoder.py", line 201, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/usr/lib/python2.7/json/encoder.py", line 264, in iterencode
return _iterencode(o, 0)
UnicodeDecodeError: 'utf8' codec can't decode byte 0x82 in position 0: invalid start byte

@toastdriven toastdriven was assigned Dec 4, 2013

I think there just needs to be some documentation in the dynamodb2 pages about how to use the types from dynamodb1 instead of changing any code here.

boto.dynamodb.types.Binary(umsgpack.packb( etc etc ))

@danielgtaylor danielgtaylor was assigned Dec 6, 2013
the boto project member

Yes, you can use the dynamodb1 types for dynamodb2. For the vast majority of data you won't need to use them directly, but since it seems that your use case requires them this makes sense. As for documentation updates, please feel free to reword or add to the existing docs to make this more obvious. We gladly accept pull requests that fix or augment the documentation 😄

I'll leave this issue open for now and you can reference it in the pull request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment