Large value insertion above thrift_max_message_length_in_mb #168

Closed
jterrace opened this Issue Jul 30, 2012 · 1 comment

2 participants

@jterrace

If you try and insert a really large value above the thrift_max_message_length_in_mb setting, you get a TTransportException: with message "TSocket read 0 bytes":

>>> import pycassa
>>> pool = pycassa.ConnectionPool(keyspace='Sirikata', server_list=['localhost:9160'], timeout=20, pool_size=1)
>>> cf = pycassa.ColumnFamily(pool, 'TempFiles')
>>> import hashlib
>>> data = 'x' * 1024 * 1024 * 200
>>> key = '514f0b9344ea04c03032092f3cc4449bbd384aa226b4f7355b39705aa99d3ad2'
>>> cf.insert(key, {'size': len(data), 'username': 'jterrace', 'chunk_list': '0', '0': data})
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.6/site-packages/pycassa/columnfamily.py", line 870, in insert
    write_consistency_level or self.write_consistency_level)
  File "/usr/lib/python2.6/site-packages/pycassa/pool.py", line 546, in execute
    return getattr(conn, f)(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/pycassa/pool.py", line 144, in new_f
    return new_f(self, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/pycassa/pool.py", line 144, in new_f
    return new_f(self, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/pycassa/pool.py", line 144, in new_f
    return new_f(self, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/pycassa/pool.py", line 144, in new_f
    return new_f(self, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/pycassa/pool.py", line 144, in new_f
    return new_f(self, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/pycassa/pool.py", line 139, in new_f
    (self._retry_count, exc.__class__.__name__, exc))
pycassa.pool.MaximumRetryException: Retried 6 times. Last failure was TTransportException: TSocket read 0 bytes
>>> 

This took me a long time to figure out what was going on. Is there any way to make this error more obvious? Not sure if it's a pycassa issue, or thrift issue, or a cassandra error issue.

@thobbs
pycassa member

Hmm, from pycassa's perspective, it can't tell the difference between this and other transport errors. Looks like it's primarily Thrift's fault server-side, as it raises a generic TException when the message is too long.

I've opened THRIFT-1665 to get the fix started. After that's in and Cassandra is using a recent version of Thrift, there may be some required changes in Cassandra to handle the different Exception type correctly.

@thobbs thobbs closed this Jul 30, 2012
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment