Handling non UTF-8 files

[cgestes]

calling replacer zsh sss in .config/ctafconf

I got the following backtrace with the last commits:

  File "/home/ctaf/.config/ctafconf/bin/replacer", line 208, in replace_in_file
    in_lines = in_fd.readlines()
  File "/usr/lib/python3.6/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 10: invalid continuation byte

[dmerejkowsky]

Oh. This is a regression introduced by this commit: aad74e9

My mistake.

The previous implementation used to just skip non-utf-8 encoding files, so I'll restore that.

Better fixes could be:

• use bytes instead of strings (recent python3 versions allow regexp on bytes) (Requires Get rid of Python2 compat #1 because life is too short)
• use chardet and preserve the encoding (if input is latin-1, output will be latin-1 too)
• add a --encoding option ?

[dmerejkowsky]

Regression fixed, keeping the issue open for discussion.

[cgestes]

Seems good enough as is.