Handle Python layer exceptions correctly #2462

Merged
merged 3 commits into from Aug 6, 2015

Conversation

Projects
None yet
2 participants
Contributor

longjon commented May 15, 2015

(This is a simplified alternative implementation of part of the first commit of #2001, plus a test.)

As noted by @tnarihi, Python layer exceptions are not handled correctly in pycaffe. PyErr_Print is used to display exception text on stderr, but this has the side effect of clearing the exception, which results in an additional SystemError: NULL return without exception set.

The exceptions get passed through correctly if we simply don't try to catch and print them; that's the first commit. (Note that the Python interpreter will print these exceptions, if uncaught; it's only the caffe tool that won't.) Since we'd still like to see the exception text when using the caffe tool, the second commit adds a single, ifdefed, catch/PyErr_Print block to the tool code. (There's no harm in calling PyErr_Print since we know we're not in the interpreter.)

I prefer this approach to the one in #2001, for these reasons:

  • it simplifies the Python wrapper code, removing the duplicate error catching blocks
  • it removes the need to think about where exceptions might occur in future wrapper code
  • it avoids the business of embedding Python calls to read (and restore) the exception
  • calling code (from C++) has the option of avoiding the exception printout (e.g., and dealing with the exception)

The disadvantage of this approach is that:

  • Python exceptions won't be automatically printed when invoking nets with Python layers from C++ code (outside the Python interpreter) other than the caffe tool

Let me know what you think @tnarihi and others, and thanks @tnarihi for pointing out this issue. (Note also that this does not include all of the exception-fixing functionality from #2001, which I'll plan to check out separately.)

longjon added some commits May 15, 2015

@longjon longjon [pycaffe] correct exceptions from Python; remove PyErr_Print
Previously, PyErr_Print was used to print Python exceptions. This has
the side effect of clearing the exception, which results in (an
additional) SystemError in the Python interpreter. Exception printing
from the caffe binary tool is re-added in a future commit.
41d13c0
@longjon longjon print Python exceptions when using Python layer with the caffe tool cebce77
@longjon longjon [pytest] check that Python receives (correct) exceptions from Python …
…layers
977023f
Owner

shelhamer commented Aug 6, 2015

Merging as I agree that this is simpler than the solution in #2001. Thanks Jon and thanks Takuya for raising the issue!

@shelhamer shelhamer added a commit that referenced this pull request Aug 6, 2015

@shelhamer shelhamer Merge pull request #2462 from longjon/correct-python-exceptions
Handle Python layer exceptions correctly
ac6d4b6

@shelhamer shelhamer merged commit ac6d4b6 into BVLC:master Aug 6, 2015

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

@ctrevino ctrevino added a commit to Robotertechnik/caffe that referenced this pull request Aug 6, 2015

@ctrevino ctrevino Merge remote-tracking branch 'upstream/master'
 Merge pull request #2462 from longjon/correct-python-exceptions
c95cc07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment