Locale fallback doesn't consider Debian's /etc/default/locale

[cgevans]

The resolution of #277 by commit 0a51b17 caused fish to silently set LC_CTYPE to en_US.UTF-8 if it does not get a locale from elsewhere, even if this is not a valid locale on the system.

The comment at the end of that commit suggests that the intention here was to set the character encoding to UTF-8, but on machines that don't have en_US (for example, outside of the US), it doesn't do that, and instead sets an invalid locale. This causes a number of problems (in my case, line editing and prompts broke subtly, as well as non-ASCII characters, as expected).

The commit does check for default locales in a few places, but not /etc/default/locale, the location that Debian and (I think) Ubuntu use.

My specific case here is connecting to a Debian 11 system (with en_IE.utf8), via SSH, from an Arch Linux system (which does not set SendEnv LANG LC_*, as Debian systems do).

I'm not sure what a better option is here than current behaviour, however:

1. It would make sense to check /etc/default/locale in order to support Debian. It appears that this would only require adding it to the list of checked files.
2. Checking locale -a to see whether en_US.UTF-8 is a supported locale before setting it might be possible, but might not be portable or cheap.
3. Setting a hardcoded US locale when a locale is not set (and most other shells would assume C and leave the locale unset) seems surprising, and does cause problems outside the US; it might make sense to reconsider this, or warn the user.

[floam]

Setting a hardcoded US locale when a locale is not set (and most other shells would assume C and leave the locale unset) seems surprising, and does cause problems outside the US; it might make sense to reconsider this, or warn the user.

I agree. It seems to me we should leave it as-is unless we can get some good information that we are just upgrading to UTF-8.

[faho]

Setting a hardcoded US locale when a locale is not set

This is not a US locale. This is a UTF-8 locale.

It's LC_CTYPE, which is the encoding value. The "en_US" bit is essentially superfluous, we just unfortunately need to pick one because it's how locales work.

The comment at the end of that commit suggests that the intention here was to set the character encoding to UTF-8, but on machines that don't have en_US (for example, outside of the US), it doesn't do that, and instead sets an invalid locale.

It does basically the same thing as having a locale of C. So if you don't have it it simply doesn't help.

This is a fallback's fallback's fallback, and it is unfortunately the best we can do until systems all grow a C.UTF-8 locale (which seems to be heading to POSIX)

[floam]

How is that not a bug? What fish is doing isn't something that should work.

[faho]

Fish can choose to either do nothing, in which case things are broken, or it can choose to do something, in which case things might be unbroken or might stay broken.

Again: This is a fallback's fallback's fallback. This is the Hail Mary. This is the thing we do when nothing else works, when the system just seems to be unable to handle 99% of unicode. We try it, and either it works, or it doesn't.

The idea that this breaks things is essentially wrong. Switching to an invalid locale has basically the same effect as staying at "C". Everything will assume ASCII, and everything will be broken with anything non-ASCII.

[floam]

Isn't this just something that happens in the situation where the locale is not already set?

[floam]

That shouldn't be an emergency.