Vim removes last newline

[LLyaudet]

Hi,

I have found the following bug. When I open a file in Vim the last newline disappears.
Hence it is impossible to see it is there and to remove it (suppress or backspace doesn't work).
The problem occurs both with Linux '\n' and Windows '\r\n'.

I join a sample text file with a '\r\n'
testVim3.txt
Another sample text file with a '\n'
testVim5.txt

The problem is quite serious for php files
testVim.txt
since additional characters after "?>" are sent as HTML and you can get an error "Headers already sent" if you want to output something else that HTML.

I had to compare the result in Nano, Notepad++ and hexdump to realize the problem was with Vim.

Best regards,
Laurent Lyaudet

[LLyaudet]

I forgot to say that I tested it on Debian:
VIM - Vi IMproved 7.3 (2010 Aug 15, compiled Feb 10 2013 02:28:47)
Rustines incluses : 1-547
and
VIM - Vi IMproved 7.4 (2013 Aug 10, compiled Mar 31 2015 23:36:03)
Rustines incluses : 1-488, 576

I also tested it on Ubuntu:
VIM - Vi IMproved 7.4 (2013 Aug 10, compiled Jan 2 2014 19:39:32)
Rustines incluses : 1-52

[chrisbra]

Sorry, I don't follow. Your file testVim5.txt consists only of a single linefeed. Why do you expect it to be shown as 2 different lines?

[mgedmin]

On Unix the \n is a line terminator, not a line separator. Every line is supposed to have a \n at the end, and vim enforces that.

See also :h 'eol' and :h 'fixeol'.

[lilydjwg]

@LLyaudet for those PHP files, just don't use ?> at the end so nothing will be output unexpectedly.

Also, to distinguish the difference, you can use the status line. My status line will show [noeol] to indicate there is one lacking \n so I know there will be problems if I use tools like cat. I also have a [BOM] indicator when the file has BOM so e.g. you don't accidentally output that in PHP files.

[chrisbra]

This has been discussed numereous times and can be found in the list archives. In short, this is not a Vim bug, but the traditional behaviour on Unix and preceeds the Windows behaviour. So closing as invalid.

[vim-ml]

Somebody mentioned it in passing; and I agree this isn't a bug, so the issue should remain closed, however:

The user has a problem. The problem can be fixed with different user settings.

You need to set noeol and nofixeol (in very recent versions of Vim) if you want to remove the last linefeed character.

There are complicated workarounds using the 'binary' option for earlier Vim versions without the fixeol option.

[vim-ml]

On Fri, Nov 20, 2015 at 4:44 PM, Ben Fritz fritzophrenic@gmail.com wrote:

Somebody mentioned it in passing; and I agree this isn't a bug, so the issue should remain closed, however:

The user has a problem. The problem can be fixed with different user settings.

You need to set noeol and nofixeol (in very recent versions of Vim) if you want to remove the last linefeed character.

There are complicated workarounds using the 'binary' option for earlier Vim versions without the fixeol option.

The workarounds are not complicated, however it must be remembered
that closing a file to edit sets 'eol' on on it, unless it has either
'binary' or (in recent Vim versions) 'nofixeol'. So, even in old Vim
builds:

• To do it once, use
  :setlocal bin noeol
  just before saving.
• Or you may set these same options by means of a modeline (q.v.) to
  make sure that the last line remains incomplete and that if ever you
  concatenate an other file to that one, the first line of the second
  file will be tacked onto the last line of the first one. :-P

Best regards,
Tony.

[LLyaudet]

Hi,
Thanks for the quick answers. I didn't knew that for Unix \n is a line terminator instead of a line separator. I have never seen this convention before. It contradicts the names "linefeed" and "newline" from which we have the convention "\n", so publicity about this is most welcome.
I have searched for sources and references about it but I have only found Wikipedia:
"Two ways to view newlines, both of which are self-consistent, are that newlines separate lines or that they terminate lines. If a newline is considered a separator, there will be no newline after the last line of a file. Some programs have problems processing the last line of a file if it is not terminated by a newline. On the other hand, programs that expect newline to be used as a separator will interpret a final newline as starting a new (empty) line."
in
https://en.wikipedia.org/wiki/Newline
I tried to search for use case of line terminators and so far I see it's simpler to concatenate files or include files in another one. There is probably other pros to it so if you can provide references I would be happy to learn.

Personally, I just see a file as a string and it's weird if the empty string is displayed the same as the string "\n". Vim shows 1 line when there is none, 1 line when there is 1, etc. Other editors show 1 line when there is none, 2 lines when there is 1 because you visualize the string plus the possible places for insertion.
So "" is in reality "|" in the editor (I use | for the possible caret), "\n" is in reality "|\n" in vim and "|\n|" in other editors.
So the fun point is that in vim, when you type '\n' in "|\n", you're adding a newline before the end, you never insert at the end (logically, probably it's hacked to concatenate anyway behind the curtain).

Anyway I read the help for eol as suggested (fixeol is not already available for my version of Vim).
I'll use fixeol when available.

@lilydjwg I knew about removing "?>" at the end but I must admit I find it ugly to have "". I'll ask to my development team what they think about it.

Best regards,
Laurent Lyaudet

[vim-ml]

On Friday, November 20, 2015 at 1:35:35 PM UTC-6, Tony Mechelynck wrote:

On Fri, Nov 20, 2015 at 4:44 PM, Ben Fritz fritzophrenic@gmail.com wrote:

There are complicated workarounds using the 'binary' option for earlier Vim versions without the fixeol option.

The workarounds are not complicated, however it must be remembered
that closing a file to edit sets 'eol' on on it, unless it has either
'binary' or (in recent Vim versions) 'nofixeol'. So, even in old Vim
builds:

• To do it once, use
  :setlocal bin noeol
  just before saving.

The complicated part comes in because 'binary' will always write with Unix line endings even if the file was read with something else. See http://vim.wikia.com/wiki/Preserve_missing_end-of-line_at_end_of_text_files for a full workaround preserving not only the missing end of line but also the fileformat.

The script there should work to accomplish the task at hand in older Vims that don't have the 'fixeol' option.

[vim-ml]

On Fri, Nov 20, 2015 at 9:14 PM, Laurent Lyaudet
vim-dev-github@256bit.org wrote:

Personally, I just see a file as a string and it's weird if the empty string is displayed the same as the string "\n". Vim shows 1 line when there is none, 1 line when there is 1, etc. Other editors show 1 line when there is none, 2 lines when there is 1 because you visualize the string plus the possible places for insertion.

Actually, Vim makes a difference: if you have 'ruler' on, or if your
'statusline' shows the current line and column (usually near its right
end), you'll see them displayed as 0,0-1 when in an empty file, but as
1,0-1 when on the only (empty) line in a file consisting only of a
linefeed (or, if 'fileformat' is set to "dos", of a crarriage return -
line feed pair). (And yes, when using papertape on a Teletype 33
almost fifty years ago I wanted the carriage to return to the left
margin and the line to be fed even after the last line in the text. I
even learned to follow every CR-LF pair with three rubouts in order to
give the carriage enough time to return to the margin.)

Best regards,
Tony.