[RFC] Default 'encoding'-option from latin to utf-8

[fornwall]

Normally the default encoding does not have much effect, since it's
overridden by the environment.

But when it's not (test with "LANG= LC_ALL= C_CTYPE= nvim" and perform
":set encoding?"), utf-8 should be the default encoding for a 21st
century editor :).

[atwupack]

Is it really necessary that Neovim supports latin1 or other non Unicode encodings at all? All platforms Neovim runs on will have Unicode support I guess. So you could use utf-8 as the standard encoding and switch to something "bigger" as utf-16 or utf-32 if something out of the utf-8 range is needed.

[rlidwka]

As a default, utf-8 is a good idea indeed.

[justinmk]

There's a remote possibility that this could affect plugins. But this note from :help 'encoding' seems to indicate that changing the default is probably fine for Neovim, because we are moving away from explicit handling of terminal:

"utf-8" has not been made the default to prevent different behavior of the GUI and terminal versions, and to avoid changing the encoding of newly created files without your knowledge (in case 'fileencodings' is empty).

On the other hand, changing the defaults isn't really a priority until we get near a release, because all of our current users are veteran Vim users who already set the encoding in their vimrc.

Is it really necessary that Neovim supports latin1 or other non Unicode encodings at all?

@atwupack Removing support for latin1 is a non-trivial change (search for "latin1" in the source). And it is still useful to be able to convert files to latin1 using [n]vim.

[fornwall]

On the other hand, changing the defaults isn't really a priority until we get near a release, because all of our current users are veteran Vim users who already set the encoding in their vimrc.

Yes, but I don't think there is any value in waiting with changing default settings to make nvim usable out of the box - the sooner they get made the better, so to close in on a usable release and have time to work out any eventual problem from switching the default values.

[atwupack]

@justinmk To have latin1 as value for fileenc is useful of course and should not be removed. But I can read/write latin1 files with internal encoding set to utf-8. Or is there some deeper correlation between fileenc and encoding that I do not understand?

[ZyX-I]

There's a remote possibility that this could affect plugins. But this note from :help 'encoding' seems to indicate that changing the default is probably fine for Neovim, because we are moving away from explicit handling of terminal:

I know that some plugins use &encoding for some encoding conversions. No details though as a) I do not use this b) it is potentially dangerous and c) I only saw some replies on SO and/or vim-use and when I saw it I usually replied that one must not use &encoding for doing conversions and there are other options (iconv(), ++enc, fileencoding: depends on what you need).

I do not think that changing default to utf-8 is harmful: lots of users set encoding to utf-8 near the top of their vimrc and even plugins with such hacks must work with this &encoding setting. If they do not they are utterly broken because there is no reason why user default should not break plugins and neovim default should since set enc&vim or set enc&vi both set encoding to utf-8 as long as I run them in UTF-8 locale.

[justinmk]

lots of users set encoding to utf-8 near the top of their vimrc and even plugins with such hacks must work with this &encoding setting

Of course, good point. 👍 LGTM

[aktau]

lots of users set encoding to utf-8 near the top of their vimrc and even plugins with such hacks must work with this &encoding setting.

Exactly right. LGTM too.

[justinmk]

merged