Invalid line length with utf8 characters

[vslinko]

Simple patch.

211c211

< length = len(physical_line.rstrip())

---

length = len(unicode(physical_line.rstrip(), 'utf-8'))

[jcrocholl]

But this will raise UnicodeDecodeError if the input file uses a different encoding. Example: With latin-1 you would have things like physical_line = "\xe4".

[jcrocholl]

We're currently using latin-1 fallback in case UTF-8 could not be decoded. A better solution would be to detect the encoding correctly, but maybe that's overkill if only for line length.

[vslinko]

I think you right.