Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mkosi: set locale to be en_US.UTF-8 #7244

Merged
merged 1 commit into from Nov 7, 2017

Conversation

@johnlinp
Copy link
Contributor

commented Nov 2, 2017

Fixes: #7238

@poettering

This comment has been minimized.

Copy link
Member

commented Nov 3, 2017

hmm, can you explain why C.UTF-8 wouldn't work here? everything I find on the internet suggests that that's a valid locale, and "local -a" appears to show it everywhere I checked as valid. I wonder if this is an ArchLinux specific thing, that that locale isn't set up properly there?

If C.UTF-8 is valid, I'd prefer to use it, since it means "don't do locale, except for using UTF-8 as character set", which is what we want and very different from "do american localization"... Given that the images we build aren't really supposed to be american, and given that we don't actually pull in localization package support (at least not where we can avoid it)

@poettering poettering added the mkosi label Nov 3, 2017

@johnlinp

This comment has been minimized.

Copy link
Contributor Author

commented Nov 5, 2017

I did a bit of research on C.UTF-8. Please correct me if I'm wrong.

I don't think C.UTF-8 is valid everywhere. At least it's not valid by default (i.e. provided by glibc). Take Fedora as an example, C.UTF-8 isn't supported before this bug is fixed.

I suggest that we can start using C.UTF-8 only after this upstream glibc bug is fixed. Thank you.

@poettering poettering merged commit 474cca4 into systemd:master Nov 7, 2017

4 of 5 checks passed

xenial-s390x autopkgtest finished (failure)
Details
Fedora Rawhide CI x86_64 rpm build succeeded
Details
semaphoreci The build passed on Semaphore.
Details
xenial-amd64 autopkgtest finished (success)
Details
xenial-i386 autopkgtest finished (success)
Details
@mbiebl

This comment has been minimized.

Copy link
Contributor

commented Nov 7, 2017

but en_US.UTF-8 is also not guaranteed to exist everywhere (even less likely then C.UTF-8)
So I'm not sure this commit is actually fixing anything. It just breaks it for a different set of people.

@mbiebl

This comment has been minimized.

Copy link
Contributor

commented Nov 7, 2017

For example, on a couple of Debian systems I have:

# locale -a
C
C.UTF-8
de_DE.utf8
POSIX

C.UTF-8 is guaranteed to be enabled on every Debian/Ubuntu based system. I think it's the same for Fedora. Other UTF-8 locales are not guaranteed to exist.

@poettering

This comment has been minimized.

Copy link
Member

commented Nov 7, 2017

urks... do we have to auto-detect what is available? yuck!

@mbiebl

This comment has been minimized.

Copy link
Contributor

commented Nov 7, 2017

@poettering I think you can only rely on C to be universally available.
C.UTF-8 is enabled by default (and not removable) by most major distros afaik. But no, atm it's not guaranteed to exist everywhere. Choosing en_US.UTF-8 is even worse.

@johnlinp

This comment has been minimized.

Copy link
Contributor Author

commented Nov 8, 2017

I don't think en_US.UTF-8 is a worse choice. Since we have mkosi.arch, mkosi.debian, mkosi.fedora in .mkosi/, then we should guarantee that systemd is able to build on these three OSs. Maybe this pull request is a workaround, but it can solve the problem for this moment.

Or, should I file another issue to wait for the glibc bug and change it back to C.UTF-8?

@poettering

This comment has been minimized.

Copy link
Member

commented Nov 8, 2017

I think the point @mbiebl was making is that en_US.UTF-8 on Debian is an optional locale, but C.UTF-8 is not there...

I figure we need to do some auto-discovery here: use C.UTF-8 when it exists, fallback to en_US.UTF-8 if it doesn't, fail if that doesn't exist too

@poettering

This comment has been minimized.

Copy link
Member

commented Nov 8, 2017

(and other than that, I think the arch folks should also do what debian and fedora appear to do: guarantee for the existance of C.UTF-8)

@mbiebl

This comment has been minimized.

Copy link
Contributor

commented Nov 8, 2017

don't think en_US.UTF-8 is a worse choice. Since we have mkosi.arch, mkosi.debian, mkosi.fedora in .mkosi/, then we should guarantee that systemd is able to build on these three OSs. Maybe this pull request is a workaround, but it can solve the problem for this moment.

It does not. As I showed you, en_US.UTF-8 is not guaranteed to exist on Debian/Ubuntu whereas C.UTF-8 is acutally guaranteed to exist.
I'm not sure if arch actually guarantees that en_US.UTF-8 always exists, but this commit broke it for Debian/Ubuntu users (unless they installed their system choosing en_US as language).

@mbiebl

This comment has been minimized.

Copy link
Contributor

commented Nov 8, 2017

I think the point @mbiebl was making is that en_US.UTF-8 on Debian is an optional locale, but C.UTF-8 is not there...

"not there" → always guaranteed to be there

@mbiebl

This comment has been minimized.

Copy link
Contributor

commented Nov 8, 2017

I figure we need to do some auto-discovery here: use C.UTF-8 when it exists, fallback to en_US.UTF-8 if it doesn't, fail if that doesn't exist too

Would mkosi support falling back to a non-UTF locale like C? If not, what you propose seems the only option then unfortunately.

@keszybz

This comment has been minimized.

Copy link
Member

commented Nov 8, 2017

Yeah, I dislike the switch too. C.UTF-8 is exactly what we should use.

@poettering

This comment has been minimized.

Copy link
Member

commented Nov 8, 2017

Would mkosi support falling back to a non-UTF locale like C? If not, what you propose seems the only option then unfortunately.

Nah, this is 2017. I'd rather make the stuff fail than end up using non-UTF8 ctypes...

filbranden added a commit to filbranden/systemd that referenced this pull request Mar 2, 2018

mkosi: use locale that supports UTF-8, detect one that is available
Using C.UTF-8 (as was done before systemd#7244) breaks Arch Linux, but using
en_US.UTF-8 (after systemd#7244) breaks Debian in our .mkosi/mkosi.debian.

So try to detect which one is available and works, first checking
whether we're already running under a valid UTF-8 locale, then trying
C.UTF-8 and finally en_US.UTF-8.

If we fail to find a valid UTF-8 locale, then fail early, instead of
letting the whole build complete only for Mesos to fail midway through
the `ninja test` step.

Tested on all of mkosi.fedora, mkosi.debian and mkosi.arch.

Fixes: systemd#7238

keszybz added a commit that referenced this pull request Mar 5, 2018

mkosi: use locale that supports UTF-8, detect one that is available (#…
…8340)

Using C.UTF-8 (as was done before #7244) breaks Arch Linux, but using
en_US.UTF-8 (after #7244) breaks Debian in our .mkosi/mkosi.debian.

So try to detect which one is available and works, first checking
whether we're already running under a valid UTF-8 locale, then trying
C.UTF-8 and finally en_US.UTF-8.

If we fail to find a valid UTF-8 locale, then fail early, instead of
letting the whole build complete only for Mesos to fail midway through
the `ninja test` step.

Tested on all of mkosi.fedora, mkosi.debian and mkosi.arch.

Fixes: #7238
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
4 participants
You can’t perform that action at this time.