Telephus eats tracebacks

[luciotorre]

When using telephus from code that uses defer.inlineCallbacks, all exceptions show a traceback that is internal to telephus, and they never show the line in my code which generated the error.

Example:

from twisted.internet import defer, reactor
from telephus.protocol import ManagedCassandraClientFactory
from telephus.client import CassandraClient
from telephus.cassandra.ttypes import KsDef, CfDef, InvalidRequestException

KEYSPACE = "test_keyspace"
CF_NAME = "cf"

def raise_error(_):
    raise Exception("i failed")

@defer.inlineCallbacks
def error():
    d = defer.Deferred()
    reactor.callLater(1, d.callback, True)
    d.addCallback(raise_error)
    yield d

@defer.inlineCallbacks
def run():
    cmanager = ManagedCassandraClientFactory()
    client = CassandraClient(cmanager, KEYSPACE)
    reactor.connectTCP("localhost", 54070, cmanager)
    yield cmanager.deferred
    ksdef = KsDef(
                name=KEYSPACE,
                strategy_class='org.apache.cassandra.locator.SimpleStrategy',
                replication_factor=1,
                cf_defs=[ CfDef(
                    keyspace=KEYSPACE,
                    name=CF_NAME,
                    column_type="Standard")
                        ])
    #yield error()
    yield client.system_add_keyspace(ksdef)

if __name__ == "__main__":
    reactor.callWhenRunning(run)
    reactor.run()

In this example, the output, when run for a second time, will be:
Unhandled error in Deferred:
Traceback (most recent call last):
Failure: telephus.cassandra.ttypes.InvalidRequestException: InvalidRequestException(why='Keyspace already exists.')

I would expect to see the line yield client.system_add_keyspace(ksdef) somewhere in the backtrace, but it is not there.

But if we uncomment the #yield error() line, backtraces show as expected.

This issue is specially bad for actions like insert, where you can get a traceback like this:

Traceback (most recent call last):
  File "/usr/lib/python2.6/dist-packages/twisted/internet/defer.py", line 1252, in put
    self.waiting.pop(0).callback(obj)
  File "/usr/lib/python2.6/dist-packages/twisted/internet/defer.py", line 318, in callback
    self._startRunCallbacks(result)
  File "/usr/lib/python2.6/dist-packages/twisted/internet/defer.py", line 424, in _startRunCallbacks
    self._runCallbacks()
  File "/usr/lib/python2.6/dist-packages/twisted/internet/defer.py", line 441, in _runCallbacks
    self.result = callback(self.result, *args, **kw)
--- <exception caught here> ---
  File "/usr/local/lib/python2.6/dist-packages/telephus/protocol.py", line 210, in _process
    d = proto.submitRequest(request)
  File "/usr/local/lib/python2.6/dist-packages/telephus/protocol.py", line 92, in submitRequest
    d = fun(*request.args)
  File "/usr/local/lib/python2.6/dist-packages/telephus/cassandra/Cassandra.py", line 730, in insert
    self.send_insert(key, column_parent, column, consistency_level)
  File "/usr/local/lib/python2.6/dist-packages/telephus/cassandra/Cassandra.py", line 741, in send_insert
    args.write(oprot)
  File "/usr/local/lib/python2.6/dist-packages/telephus/cassandra/Cassandra.py", line 4044, in write
    oprot.trans.write(fastbinary.encode_binary(self, (self.__class__, self.thrift_spec)))
exceptions.TypeError: expected string or Unicode object, int found

and never know which part of your code failed.

[driftx]

This is an artifact of your use of inlineCallbacks. It's a problem with twisted, not telephus. Don't use inlineCallbacks if you don't like this behavior.

[luciotorre]

It is not true that if you use inlineCallbacks you will lose every traceback. See the example were i mix normal deferred usage with inlineCallbacks and you do get a traceback.

It is not an option for us to drop inlineCallbacks. We use it extensively on our software and we still get proper tracebacks when something occurs.

File "/usr/lib/python2.6/dist-packages/twisted/internet/defer.py", line 891, in _inlineCallbacks
  result = result.throwExceptionIntoGenerator(g)
File "/usr/lib/python2.6/dist-packages/twisted/python/failure.py", line 338, in throwExceptionIntoGenerator
  return g.throw(self.type, self.value, self.tb)
File "telephus_test.py", line 34, in run
  yield error()
File "/usr/lib/python2.6/dist-packages/twisted/internet/defer.py", line 891, in _inlineCallbacks
  result = result.throwExceptionIntoGenerator(g)
File "/usr/lib/python2.6/dist-packages/twisted/python/failure.py", line 338, in throwExceptionIntoGenerator
  return g.throw(self.type, self.value, self.tb)
File "telephus_test.py", line 17, in error
  yield d
File "/usr/lib/python2.6/dist-packages/twisted/internet/defer.py", line 441, in _runCallbacks
  self.result = callback(self.result, *args, **kw)
File "telephus_test.py", line 10, in raise_error
  raise Exception("i failed")
exceptions.Exception: i failed

[thepaul]

I contend that the real problem is both in Thrift's twisted code (for not starting with meaningful tracebacks) and Twisted's error handling (for being overeager in excising parts of tracebacks between calls to Failure._findFailure and Failure.raiseException, etc). Still, I think the change I just made on the unstable branch (see cb812ec) ought to help you. Try it out.