Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

STORM-3164: Fix usage of traceback.format_exc in multilang #2780

Merged
merged 1 commit into from
Jul 30, 2018

Conversation

mal
Copy link
Contributor

@mal mal commented Jul 29, 2018

Misuse of traceback.format_exc was causing exceptions thrown in python bolts to throw TypeErrors during attempts to handle them. Fixing the calls results in expect behaviour.

Examples of the change can be seen below where the Storm side messages have been mocked so as to directly inspect the python side response. Lines prefixed with # are the prompt, > are stdin (manually entered) and > denotes stdout. Unprefixed lines are stderr.

https://issues.apache.org/jira/browse/STORM-3164

Test Code

from storm import Bolt


class TestBolt(Bolt):
    def process(self, tup):
        raise Exception('harakiri!')


if __name__ == '__main__':
    TestBolt().run()
while true; do
  printf '> ' > /dev/tty
  read line
  echo "$line"
  sleep 0.1
  printf '\n' # trigger sigpipe if python is dead
done | python3 multilang/resources/bolt.py | sed 's/^/< /'

Before

# sh ./bolttest
> {"conf": {}, "context": {}, "pidDir": "/tmp"}
> end
< {"pid": 10643}
< end
> {"id":1, "comp": 2, "stream": 3, "task": 4, "tuple": ["foo", "bar"]}
> end
Traceback (most recent call last):
  File "/usr/src/test/storm.py", line 198, in run
    self.process(tup)
  File "bolt.py", line 6, in process
    raise Exception('harakiri!')
Exception: harakiri!

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "bolt.py", line 10, in <module>
    TestBolt().run()
  File "/usr/src/test/storm.py", line 200, in run
    reportError(traceback.format_exc(e))
  File "/usr/lib/python3.4/traceback.py", line 256, in format_exc
    return "".join(format_exception(*sys.exc_info(), limit=limit, chain=chain))
  File "/usr/lib/python3.4/traceback.py", line 181, in format_exception
    return list(_format_exception_iter(etype, value, tb, limit, chain))
  File "/usr/lib/python3.4/traceback.py", line 153, in _format_exception_iter
    yield from _format_list_iter(_extract_tb_iter(tb, limit=limit))
  File "/usr/lib/python3.4/traceback.py", line 18, in _format_list_iter
    for filename, lineno, name, line in extracted_list:
  File "/usr/lib/python3.4/traceback.py", line 58, in _extract_tb_or_stack_iter
    while curr is not None and (limit is None or n < limit):
TypeError: unorderable types: int() < Exception()

After

# sh ./bolttest
> {"conf": {}, "context": {}, "pidDir": "/tmp"}
> end
< {"pid": 10644}
< end
> {"id":1, "comp": 2, "stream": 3, "task": 4, "tuple": ["foo", "bar"]}
> end
< {"command": "error", "msg": "Traceback (most recent call last):\n  File \"/usr/src/test/storm.py\", line 198, in run\n    self.process(tup)\n  File \"bolt.py\", line 6, in process\n    raise Exception('harakiri!')\nException: harakiri!\n"}
< end

Copy link
Contributor

@HeartSaVioR HeartSaVioR left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 Wondering why we miss this so long. Nice catch!

@revans2
Copy link
Contributor

revans2 commented Jul 30, 2018

+1

@asfgit asfgit merged commit 5ed30b5 into apache:master Jul 30, 2018
asfgit pushed a commit that referenced this pull request Jul 30, 2018
…3164

STORM-3164: Fix usage of traceback.format_exc in multilang

This closes #2780
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants