ENH: Clearer warning with lines which failed to decode correctly #723
Conversation
| (self.getFileName(), self.getEncoding(), `line`)) | ||
| logSys.warning( | ||
| "Error decoding line from '%s' with '%s'. Continuing " | ||
| " to process line ignoring invalid characters: %r" % |
There was a problem hiding this comment.
well... with python2 it would not decode line at all, not just ignore those invalid characters... btw -- I wondered why in d23d365 you added decoding with 'ignore' only for python3 and not also for python2 (seems to be available there too)? ;-)
There was a problem hiding this comment.
I believe at the time, the idea was trying to keep behaviour the same as 0.8, but this wasn't possible in Python 3. I think however, that it would be better for Python 2 version to have the same behaviour as Python 3, as this keeps it consistent, and probably is the behaviour users would expect when specifying a encoding.
ENH: Clearer warning with lines which failed to decode correctly
|
Thanks for merging @yarikoptic. Did you have any thoughts about changing the code to have same behaviour in both Python 2 and 3? |
This is to make it clear that lines are still processed, even if strict decoding fails. Ref #718