Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dynamodb2.table.Table.batch_write promotes str to unicode incorrectly. #1889

Open
jschachter opened this issue Dec 3, 2013 · 2 comments
Open

Comments

@jschachter
Copy link

it appears that you can't use batch_write to load binary data.

r = { 'word': 'yes'}
r['s1'] = umsgpack.packb( { 's1': True } )

pprint.pprint(r)
print type(r['s1'])
batch.put_item( data=r )

...
{'s1': '\x82\xc4\x01s\xc3\xc4\x01t\xc3', 'word': 'yes'}
<type 'str'>
Traceback (most recent call last):
File "test-dynamo", line 39, in
batch.put_item( data=r )
File "/usr/local/lib/python2.7/dist-packages/boto/dynamodb2/table.py", line 1083, in exit
self.flush()
File "/usr/local/lib/python2.7/dist-packages/boto/dynamodb2/table.py", line 1126, in flush
self.table.connection.batch_write_item(batch_data)
File "/usr/local/lib/python2.7/dist-packages/boto/dynamodb2/layer1.py", line 266, in batch_write_item
body=json.dumps(params))
File "/usr/lib/python2.7/json/init.py", line 231, in dumps
return _default_encoder.encode(obj)
File "/usr/lib/python2.7/json/encoder.py", line 201, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/usr/lib/python2.7/json/encoder.py", line 264, in iterencode
return _iterencode(o, 0)
UnicodeDecodeError: 'utf8' codec can't decode byte 0x82 in position 0: invalid start byte

@jschachter
Copy link
Author

I think there just needs to be some documentation in the dynamodb2 pages about how to use the types from dynamodb1 instead of changing any code here.

boto.dynamodb.types.Binary(umsgpack.packb( etc etc ))

@ghost ghost assigned danielgtaylor Dec 6, 2013
@danielgtaylor
Copy link
Member

Yes, you can use the dynamodb1 types for dynamodb2. For the vast majority of data you won't need to use them directly, but since it seems that your use case requires them this makes sense. As for documentation updates, please feel free to reword or add to the existing docs to make this more obvious. We gladly accept pull requests that fix or augment the documentation 😄

I'll leave this issue open for now and you can reference it in the pull request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants