sys.excepthook not always firing

[ericzundel]

In our repo running from a custom .pex installation of pants, we've noticed stacktraces coming out when we don't expect them. When I went in to debug this, I noticed that if you pass a bad spec on the cmdline, the _unhandled_exception_hook() method doesn't get run.

Here's how to reproduce it:

First, from a clean mastter branch, edit src/python/pants/bin/pants_exe.py and add a print stmt to _unhandled_exception_hook() as follows:

def _unhandled_exception_hook(exception_class, exception, tb):
  print ("UNHANDLED_EXCEPTION_HOOK")  
    ...

Now, execute pants in dev mode with a bogus target to see that it is working:

PANTS_DEV=1 ./pants compile foo/bar
*** Running pants in dev mode from /Users/zundel/Src/Pants/src/python/pants/bin/pants_exe.py ***
UNHANDLED_EXCEPTION_HOOK

Exception message: Path to BUILD file does not exist at: /Users/zundel/Src/pants/foo

Now, run the same command, but this time from a pex:

git clean -fdx
./pants binary ./src/python/pants/bin:pants_local_binary
./dist/pants_local_binary.pex compile foo/bar
Traceback (most recent call last):
  File "/Users/zundel/Src/pants/dist/pants_local_binary.pex/.bootstrap/_pex/pex.py", line 272, in execute
    self.execute_entry(entry_point, args)
  File "/Users/zundel/Src/pants/dist/pants_local_binary.pex/.bootstrap/_pex/pex.py", line 320, in execute_entry
    runner(entry_point)
  File "/Users/zundel/Src/pants/dist/pants_local_binary.pex/.bootstrap/_pex/pex.py", line 343, in execute_pkg_resources
    runner()
  File "/Users/zundel/Src/pants/dist/pants_local_binary.pex/pants/bin/pants_exe.py", line 67, in main
    _run()
  File "/Users/zundel/Src/pants/dist/pants_local_binary.pex/pants/bin/pants_exe.py", line 61, in _run
    goal_runner.setup()
  File "/Users/zundel/Src/pants/dist/pants_local_binary.pex/pants/bin/goal_runner.py", line 107, in setup
    self._expand_goals_and_specs()
  File "/Users/zundel/Src/pants/dist/pants_local_binary.pex/pants/bin/goal_runner.py", line 157, in _expand_goals_and_specs
    for address in spec_parser.parse_addresses(spec, fail_fast):
  File "/Users/zundel/Src/pants/dist/pants_local_binary.pex/pants/base/cmd_line_spec_parser.py", line 84, in parse_addresses
    for address in self._parse_spec(spec, fail_fast):
  File "/Users/zundel/Src/pants/dist/pants_local_binary.pex/pants/base/cmd_line_spec_parser.py", line 171, in _parse_spec
    raise self.BadSpecError(e)
BadSpecError: Path to BUILD file does not exist at: /Users/zundel/Src/pants/foo

It appears that the method we passed to sys.excepthook isn't being called!

[jsirois]

This will require a fix in pex itself. Right now it forceably traps any exception and prints its backtrace before turning off the sys.excepthook in a finally. It should probably directly invoke the sys.excepthook that's installed: https://github.com/pantsbuild/pex/blob/master/pex/pex.py#L276

[jsirois]

https://rbcommons.com/s/twitter/r/1733/

[jsirois]

Next step is to ingest pex 0.8.6 when its released. After that our custom except hook needs work - as-is, when its fired both sys and traceback modules are gone even though they were imported at the top of the file. Adding imports inside the excepthook fixes this. The same happens for the helper functions the hook currently uses - they are None as well when hook execution time rolls around. I found no way to recover these globals but inlining the code of course worked fine.

[ericzundel]

Thanks John!

On Sat, Feb 7, 2015, 7:10 PM John Sirois notifications@github.com wrote:

Next step is to ingest pex 0.8.6 when its released. After that our custom
except hook needs work - as-is, when its fired both sys and traceback
modules are gone even though they were imported at the top of the file.
Adding imports inside the excepthook fixes this. The same happens for the
helper functions the hook currently uses - they are None as well when hook
execution time rolls around. I found no way to recover these globals but
inlining the code of course worked fine.

—
Reply to this email directly or view it on GitHub
#1026 (comment).

[jsirois]

https://rbcommons.com/s/twitter/r/1734/

[jsirois]

Last step is to upgrade to pex 0.8.6 when it's released. That release is tracked here: pex-tool/pex#43

[jsirois]

Upgraded @ 6d4d1f5