Maintain OS specific end-of-lines in code files.

[carlosperate]

Possibly fixes #112 and #85.

At the moment we are having issues processing the Windows end of line format (\r\n). In essence scintilla automatically selects the host OS line ending type (as it should), so we have the editor producing \r\n endings, and the default saving option in python tries to convert to the native type, replacing each \n with \r\n, and causing the final format saved as \r\r\n. By the default open options, which try to convert windows line endings into single \n, this is then read back as \n\n.

We should definitely keep compatibility with the Windows format for both opening and saving files. So the idea is to keep scintilla appending \r\n, save the files as they are edited without any modifications, and opening them without converting the windows line ending into single \n either.

So I've added the newline=\n argument to the each open() used to write the editor file. This sets the default line endings as \n, which does not replace \n\r with \n, it just leaves them alone. The open() calls for reading files have been configured to newline=os.linesep, which should maintain the same behaviour on linux/osx and not translate the windows \r\n into \n.

I have left the settings file alone as all the new lines produced by Python itself should be \n and correctly converted. However we should note that the log file currently copies logs the code every time the file is saved, so this could be revisited in the future.

So, a general question I’ve got is if we are maintining Python2 compatibility at all? The open(newline=) argument is Python3 only.

[carlosperate]

I had an original missunderstanding of how the newline argument worked. Having a second look at the documentation it is now clear they should all be '':

On input, if newline is None, universal newlines mode is enabled. Lines in the input can end in '\n', '\r', or '\r\n', and these are translated into '\n' before being returned to the caller. If it is '', universal newline mode is enabled, but line endings are returned to the caller untranslated. If it has any of the other legal values, input lines are only terminated by the given string, and the line ending is returned to the caller untranslated.

On output, if newline is None, any '\n' characters written are translated to the system default line separator, os.linesep. If newline is '', no translation takes place. If newline is any of the other legal values, any '\n' characters written are translated to the given string.

[ntoll]

Great stuff and many thanks!