End of line (EOL) Bear

[jayvdb]

It would be nice to have a bear that required a consistent end-of-line (EOL) marker, which is often missing at the EOF.

It should work even if allow_trailing_whitespace is enabled.

[gitmate-bot]

Thanks for reporting this issue! A coalaian will look at it soon.

[sils]

Ah like unix vs. linux eol markers?

Usually git takes care of autoconverting those if it's not misconfigured. I think when we read the file into python those get normalized but if there's an important usecase we can probably find a way to make this happen

[jayvdb]

The scenario I have is that there is at least one file with DOS EOL, while most of the repository is Unix EOL.
git clone with mostly default git config didnt automatically convert it to Unix EOL.

[yp-palF]

Please help me understanding what actually is wanted. I mean what does bear that required a consistent end-of-line (EOL) marker. means.

[Makman2]

On Windows line endings are indicated with CRLF (carriage return + line feed), while linux uses LF (line feed) only. The bear shall just check / enforce the uniformity of a single end-of-line (EOL) marker (CRLF or LF).

[yp-palF]

Thanks @Makman2, So now as I can understand I have got to find \n (LF) and \r\n (CRLF) in a file and need to report the presence of other (in case of check) and change to any one kind (in case of enforce). Am I correct?

[jayvdb]

Correct.
Ideally for enforce, there is an option to specify what EOL should be used, defaulting to unix style.
It also would help to have a "per-OS" setting, but that is something that can be added after.

[yp-palF]

While working up I came to know that when python opens a file it converts '\r\n' to '\n' by default. See the second answer http://stackoverflow.com/questions/4599936/handling-r-n-vs-n-newlines-in-python-on-mac-vs-windows
any ideas how to get over this ?

[jayvdb]

Open the file in binary mode.

[jayvdb]

Or depending on how your are implementing this EOLBear, you might use the Python 3 only open newline parameter (http://stackoverflow.com/a/17562481/5037965) so that one (default) NL mode is natively handled by open, and you can see the other types of newline.

[sils]

actually @Udayan12167 has a PR introducing a FileProxy, which could on demand give you the binary version of the files or something so you don't need to read the file again? @Udayan12167 ?

[yp-palF]

alright! so I should wait for merge on the PR or I should try by reading file again?

[sils]

Please speak with udayan about this

On 3 Oct 2016 13:42, "Saurav Mehrotra" notifications@github.com wrote:

alright! so I should wait for merge on the PR or I should try by reading
file again?

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#815 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AFc6KB2K2C1FxMspkw0tLA_rjPhr5_BNks5qwPhFgaJpZM4J5-wN
.

[Adrianzatreanu]

ping @Flap-Py

[yp-palF]

Is the fileproxy done. I was waiting for that. And also can I get a sort of walk-through about how to use it?

[jayvdb]

Something I found while doing rocky/python-filecache#3 , it is possible to know that the EOLs are inconsistent while using universal new lines mode.
The file handle contains an attribute newlines that provides a list of the EOL markers found while reading the file. If the file has only a single consistent EOL, it will have only one value in newlines. If there are inconsistent line endings, it will contain a tuple of the EOLs found.

So, without loading a separate copy of the file with alternative line endings, we can know that the file has inconsistent line endings, but we do not know which lines were using a different EOL. IMO that is sufficient, because we can also auto fix the file to have a consistent EOL.
This is also easier to integrate into the design for coala/cEPs#31 , as it doesnt require a separate and mostly unusable copy of the file contents.

[jayvdb]

While waiting for a perfect solution, a decent solution is to have a dos2unix 'linter' bear.
The 'quickest' mode is modifying the files in-place -- need to see if there is a way to only update timestamps if the file was modified.

[jayvdb]

And there are some file format which should be DOS EOL even in a Unix repo, such as Batch files. c.f. review comments on loklak/loklak_server#904