Fix printing non-(ASCII/UTF-8) strings

[ghost]

As reported in FreeBSD's PR241491, ps(1) using libxo does not display the
date in uk_UA.KOI8-U locale when invoked as ps -o lstart and prints
empty lines instead. To fix it we need to assume that %s is passed in
current locale encoding.

• use XF_ENC_LOCALE for 's' format character
• pass "" to setlocale() so that if the consumer actually called setlocale()
  itself we don't overwrite already set locale; current order of environment
  variables differs from at least FreeBSD's setlocale() where the following
  order is used:
  LC_ALL
  LC_*
  LANG

[philshafer]

On Oct 25, 2019, at 6:06 PM, Yuri Pankov ***@***.***> wrote: To fix it we need to assume that %s is passed in current locale encoding.

The “%s” format does not mean “current locale”. If ps wants that, it needs to use “%hs”. “%s” means UTF-8: https://libxo.readthedocs.io/en/latest/field-formatting.html#utf-8-and-locale-strings <https://libxo.readthedocs.io/en/latest/field-formatting.html#utf-8-and-locale-strings> Re: the admittedly strange arg to setlocale(), my testing found that “” didn’t pick up the environment settings, so I coded it explicitly. If the order doesn’t follow the FreeBSD specs, I can repair it. Is this order documented somewhere in the FreeBSD docs? Thanks, Phil

[ghost]

On Oct 25, 2019, at 6:06 PM, Yuri Pankov @.***> wrote: To fix it we need to assume that %s is passed in current locale encoding.
The “%s” format does not mean “current locale”. If ps wants that, it needs to use “%hs”. “%s” means UTF-8: https://libxo.readthedocs.io/en/latest/field-formatting.html#utf-8-and-locale-strings https://libxo.readthedocs.io/en/latest/field-formatting.html#utf-8-and-locale-strings

Got it, and sorry, that the first time I looked at libxo specific issue. I'll close this one and will fix the ps instead, thanks!

Re: the admittedly strange arg to setlocale(), my testing found that “” didn’t pick up the environment settings, so I coded it explicitly. If the order doesn’t follow the FreeBSD specs, I can repair it. Is this order documented somewhere in the FreeBSD docs?

Not that I know of, the code for reference: https://svnweb.freebsd.org/base/head/lib/libc/locale/setlocale.c?revision=326025&view=markup#l285.

And if you could provide a bit more info on "" not working (possibly as PR), that would be helpful!

[philshafer]

On Oct 26, 2019, at 11:30 AM, Phil Shafer ***@***.***> wrote: Re: the admittedly strange arg to setlocale(), my testing found that “” didn’t pick up the environment settings, so I coded it explicitly. If the order doesn’t follow the FreeBSD specs, I can repair it. Is this order documented somewhere in the FreeBSD docs?

Yuri, Please review the attached patch, which mungs the order into the FreeBSD one and avoids it completely for FreeBSD. Thanks, Phil ———————— +++ b/libxo/libxo.c @@ -624,13 +624,18 @@ xo_init_handle (xo_handle_t *xop) if (!xo_locale_inited) { xo_locale_inited = 1; /* Only do this once */ - const char *cp = getenv("LC_CTYPE"); +#ifdef __FreeBSD__ /* Who does The Right Thing */ + const char *cp = ""; +#else /* __FreeBSD__ */ + const char *cp = getenv("LC_ALL"); if (cp == NULL) - cp = getenv("LANG"); + cp = getenv("LC_CTYPE"); if (cp == NULL) - cp = getenv("LC_ALL"); + cp = getenv("LANG"); if (cp == NULL) cp = "C"; /* Default for C programs */ +#endif /* __FreeBSD__ */ + (void) setlocale(LC_CTYPE, cp); }