Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace FailureFormatter with direct failure to exc_info conversions in log calls #1236

merged 2 commits into from May 15, 2015


Copy link

@curita curita commented May 15, 2015

logging.Filters can't override log messages in logger children, they can only do so in the logger they are attached. That's why FailureFormatter (the filter that takes care of extracting the exc_info from failures to print tracebacks) doesn't propagate to the descendants of the scrapy logger.

(TopLevelFormatter works because it doesn't override the "entire" log message, just the logger name, which is formatted at the end with string like '%(name)s %(msg)'. It's only the msg variable that is compute on the first logger and then propagated to the parents as it is).

I fixed it by converting each failure to exc_info before making the log call.

This pr fixes #1229 and fixes #1230.

@curita curita force-pushed the curita:fix-failure-tracebacks branch from 20d2514 to 6dccb3a May 15, 2015
def failure_to_exc_info(failure):
"""Extract exc_info from Failure instances"""
if isinstance(failure, Failure):
return (failure.type, failure.value, failure.tb)

This comment has been minimized.


dangra May 15, 2015

should we use failure.getTracebackObject() instead of failure.tb ?

This comment has been minimized.


curita May 15, 2015
Author Member

Yes, I just changed it.

@dangra dangra added this to the Scrapy 1.0 milestone May 15, 2015
@dangra dangra added the bug label May 15, 2015
dangra added a commit that referenced this pull request May 15, 2015
Replace FailureFormatter with direct failure to exc_info conversions in log calls
@dangra dangra merged commit 3a4d57b into scrapy:master May 15, 2015
1 check was pending
1 check was pending
continuous-integration/travis-ci/pr The Travis CI build is in progress
@curita curita deleted the curita:fix-failure-tracebacks branch May 19, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
None yet
Linked issues

Successfully merging this pull request may close these issues.

2 participants
You can’t perform that action at this time.