Skip to content
This repository

dynamodb2.table.Table.batch_write promotes str to unicode incorrectly. #1889

Open
jschachter opened this Issue · 2 comments

3 participants

jschachter Daniel G. Taylor Daniel Lindsley
jschachter

it appears that you can't use batch_write to load binary data.

r = { 'word': 'yes'}
r['s1'] = umsgpack.packb( { 's1': True } )

pprint.pprint(r)
print type(r['s1'])
batch.put_item( data=r )

...
{'s1': '\x82\xc4\x01s\xc3\xc4\x01t\xc3', 'word': 'yes'}

Traceback (most recent call last):
File "test-dynamo", line 39, in
batch.put_item( data=r )
File "/usr/local/lib/python2.7/dist-packages/boto/dynamodb2/table.py", line 1083, in exit
self.flush()
File "/usr/local/lib/python2.7/dist-packages/boto/dynamodb2/table.py", line 1126, in flush
self.table.connection.batch_write_item(batch_data)
File "/usr/local/lib/python2.7/dist-packages/boto/dynamodb2/layer1.py", line 266, in batch_write_item
body=json.dumps(params))
File "/usr/lib/python2.7/json/init.py", line 231, in dumps
return _default_encoder.encode(obj)
File "/usr/lib/python2.7/json/encoder.py", line 201, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/usr/lib/python2.7/json/encoder.py", line 264, in iterencode
return _iterencode(o, 0)
UnicodeDecodeError: 'utf8' codec can't decode byte 0x82 in position 0: invalid start byte

jschachter

I think there just needs to be some documentation in the dynamodb2 pages about how to use the types from dynamodb1 instead of changing any code here.

boto.dynamodb.types.Binary(umsgpack.packb( etc etc ))

Daniel G. Taylor

Yes, you can use the dynamodb1 types for dynamodb2. For the vast majority of data you won't need to use them directly, but since it seems that your use case requires them this makes sense. As for documentation updates, please feel free to reword or add to the existing docs to make this more obvious. We gladly accept pull requests that fix or augment the documentation :smile:

I'll leave this issue open for now and you can reference it in the pull request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.