A catch-all exception handler around the main file processing loop. #41
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thanks a lot! This immediately highlights another advantage of submitting a PR vs. a diff - it causes the automatic CI to run :) |
| sorted_messages_size) | ||
| except: | ||
| exc_type, exc_value, exc_traceback = exc_info() | ||
| logger.error("\n# # # # # # # EXCEPTION # # # # # # #\nType: %s\nValue: %s\nFiles parsed: %s\n# # # # # # # # # # # # # # # # # # #\n", exc_type, exc_value, message_files, exc_info=True) |
There was a problem hiding this comment.
This line is causing Travis CI to fail with:
./maildir_deduplicate/deduplicate.py:283:80: E501 line too long (202 > 79 characters)
This is easily fixable, e.g. something like
logger.error(
"\n# # # # # # # EXCEPTION # # # # # # #\n"
"Type: %s\nValue: %s\nFiles parsed: %s\n"
"# # # # # # # # # # # # # # # # # # #\n",
exc_type, exc_value, message_files, exc_info=True)
(untested) should do it.
|
Uh-oh, now I might have put you in a situation you didn't want - where you need to amend a PR :-/ If you want to do it, this might help: Otherwise feel free to give up and I can close this one and submit a new version myself :) |
This improves the situation detailed in issue #39 in so far as that exceptions no longer lead to the abortion of the entire processing batch. Exceptions are output to stderr with stack trace and the filenames being processed, so that the underlying reason and the potentially faulty files can be investigated. The execution of this is not pretty. Ideally, exception handlers should be limited to the parts of the code which actually throw exceptions, and exceptions should be handled by name. This was a quick job to stop the software from needlessly aborting the operation. The patch was written against the current stable from pip, but applied cleanly to the current development version. ---- Amended to change the formatting of the log message, because Travis CI is stuck in the 90s (80 char width limit). I have not actually tested this change. I'm sure Travis will complain anew if the syntax is off.
Current coverage is 61.41% (diff: 38.70%)@@ develop #41 diff @@
==========================================
Files 3 3
Lines 319 324 +5
Methods 0 0
Messages 0 0
Branches 71 71
==========================================
+ Hits 197 199 +2
- Misses 91 94 +3
Partials 31 31
|
…id-ass Travis won't quit crying about functionally irrelevant shit. A warning is one thing. Failing a build over violated style guidelines is stupid.
krig
added a commit
to krig/maildir-deduplicate
that referenced
this pull request
Jan 11, 2017
Some email may contain illegal characters according to the
encoding they specify.
Example error generated:
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 3701: ordinal not in range(128)
With this change, any sets with mail that have errors like that
are skipped and counted in a separate rejection bin.
This at least lets the deduplication process continue processing
all the mail.
Resolves: kdeldycke#39
Resolves: kdeldycke#41
krig
added a commit
to krig/maildir-deduplicate
that referenced
this pull request
Jan 11, 2017
Some email may contain illegal characters according to the
encoding they specify.
Example error generated:
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 3701: ordinal not in range(128)
With this change, any sets with mail that have errors like that
are skipped and counted in a separate rejection bin.
This at least lets the deduplication process continue processing
all the mail.
Resolves: kdeldycke#39
Resolves: kdeldycke#41
|
This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. |
kdeldycke
added
✨ enhancement
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This improves the situation detailed in issue #39 in so far as that exceptions no longer lead to the abortion of the entire processing batch.
Exceptions are output to stderr with stack trace and the filenames being processed, so that the underlying reason and the potentially faulty files can be investigated.
The execution of this is not pretty.
Ideally, exception handlers should be limited to the parts of the code which actually throw exceptions, and exceptions should be handled by name.
This was a quick job to stop the software from needlessly aborting the operation.
The patch was written against the current stable from pip, but applied cleanly to the current development version.