Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

setlocale's return value is broken #16435

Closed
p5pRT opened this issue Feb 23, 2018 · 22 comments
Closed

setlocale's return value is broken #16435

p5pRT opened this issue Feb 23, 2018 · 22 comments

Comments

@p5pRT
Copy link

@p5pRT p5pRT commented Feb 23, 2018

Migrated from rt.perl.org#132901 (status was 'resolved')

Searchable as RT132901$

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 23, 2018

From @eserte

This is a bug report for perl from slaven@​rezic.de,
generated with the help of perlbug 1.41 running under perl 5.27.9.


On Linux systems (checked CentOS6 and various Debians) the
following call returns binary garbage instead of the current locale​:

  $ env LC_ALL=de_DE.UTF-8 perl5.27.9 -MPOSIX=setlocale,LC_ALL -e 'warn setlocale(LC_ALL)'

This does not seem to happen if the current locale is C. And I cannot
reproduce the problem on FreeBSD systems.

The problem does not happen with 5.27.8-157-gef80cd9 or earlier perl
versions.

(Discovered because the test suite of MSCHOUT/Math-Currency-0.52.tar.gz
fails because of this problem).



Flags​:
  category=library
  severity=low
  module=POSIX


Site configuration information for perl 5.27.9​:

Configured by eserte at Tue Feb 20 21​:59​:42 CET 2018.

Summary of my perl5 (revision 5 version 27 subversion 9) configuration​:
 
  Platform​:
  osname=linux
  osvers=3.16.0-4-amd64
  archname=x86_64-linux
  uname='linux cabulja 3.16.0-4-amd64 #1 smp debian 3.16.51-3 (2017-12-13) x86_64 gnulinux '
  config_args='-ds -e -Dprefix=/opt/perl-5.27.9 -Dusedevel -Dusemallocwrap=no -Dcf_email=srezic@​cpan.org'
  hint=recommended
  useposix=true
  d_sigaction=define
  useithreads=undef
  usemultiplicity=undef
  use64bitint=define
  use64bitall=define
  uselongdouble=undef
  usemymalloc=n
  default_inc_excludes_dot=define
  bincompat5005=undef
  Compiler​:
  cc='cc'
  ccflags ='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2'
  optimize='-O2'
  cppflags='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
  ccversion=''
  gccversion='4.9.2'
  gccosandvers=''
  intsize=4
  longsize=8
  ptrsize=8
  doublesize=8
  byteorder=12345678
  doublekind=3
  d_longlong=define
  longlongsize=8
  d_longdbl=define
  longdblsize=16
  longdblkind=3
  ivtype='long'
  ivsize=8
  nvtype='double'
  nvsize=8
  Off_t='off_t'
  lseeksize=8
  alignbytes=8
  prototype=define
  Linker and Libraries​:
  ld='cc'
  ldflags =' -fstack-protector-strong -L/usr/local/lib'
  libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/4.9/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
  libs=-lpthread -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat
  perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
  libc=libc-2.19.so
  so=so
  useshrplib=false
  libperl=libperl.a
  gnulibc_version='2.19'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs
  dlext=so
  d_dlsymun=undef
  ccdlflags='-Wl,-E'
  cccdlflags='-fPIC'
  lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector-strong'


@​INC for perl 5.27.9​:
  /opt/perl-5.27.9/lib/site_perl/5.27.9/x86_64-linux
  /opt/perl-5.27.9/lib/site_perl/5.27.9
  /opt/perl-5.27.9/lib/5.27.9/x86_64-linux
  /opt/perl-5.27.9/lib/5.27.9


Environment for perl 5.27.9​:
  HOME=/home/eserte
  LANG=en_US.UTF-8
  LANGUAGE (unset)
  LD_LIBRARY_PATH (unset)
  LOGDIR (unset)
  PATH=/usr/local/bin​:/usr/bin​:/bin​:/usr/local/sbin​:/usr/sbin​:/sbin​:/home/eserte/bin/linux-gnu​:/home/eserte/bin/sh​:/home/eserte/bin​:/home/eserte/bin/pistachio-perl/bin​:/usr/games​:/home/eserte/devel
  PERLDOC=-MPod​::Perldoc​::ToTextOverstrike
  PERL_BADLANG (unset)
  SHELL=/bin/zsh

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 23, 2018

From @khwilliamson

On 02/23/2018 12​:33 AM, slaven@​rezic.de wrote​:

# New Ticket Created by slaven@​rezic.de
# Please include the string​: [perl #132901]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=132901 >

This is a bug report for perl from slaven@​rezic.de,
generated with the help of perlbug 1.41 running under perl 5.27.9.

-----------------------------------------------------------------
On Linux systems (checked CentOS6 and various Debians) the
following call returns binary garbage instead of the current locale​:

 $ env LC\_ALL=de\_DE\.UTF\-8 perl5\.27\.9 \-MPOSIX=setlocale\,LC\_ALL \-e 'warn setlocale\(LC\_ALL\)'

I don't get garbage, but I'm not getting the shortest output possible on
Ubuntu, which I have a fix for.

Please run the following on a DEBUGGING build, and post the output.
(The problem will not occur on systems that have querylocale(), nor in
locales, such as C, where the radix character is a dot)

env LC_ALL=de_DE.UTF-8 env PERL_DEBUG_LOCALE_INIT=1 perl5.27.9 -DLv
-MPOSIX=setlocale,LC_ALL -e 'warn setlocale(LC_ALL)'

(I don't know the proper syntax of stringing two env's together, so
adjust if necessary)

This does not seem to happen if the current locale is C. And I cannot
reproduce the problem on FreeBSD systems.

The problem does not happen with 5.27.8-157-gef80cd9 or earlier perl
versions.

(Discovered because the test suite of MSCHOUT/Math-Currency-0.52.tar.gz
fails because of this problem).

-----------------------------------------------------------------
---
Flags​:
category=library
severity=low
module=POSIX
---
Site configuration information for perl 5.27.9​:

Configured by eserte at Tue Feb 20 21​:59​:42 CET 2018.

Summary of my perl5 (revision 5 version 27 subversion 9) configuration​:

Platform​:
osname=linux
osvers=3.16.0-4-amd64
archname=x86_64-linux
uname='linux cabulja 3.16.0-4-amd64 #1 smp debian 3.16.51-3 (2017-12-13) x86_64 gnulinux '
config_args='-ds -e -Dprefix=/opt/perl-5.27.9 -Dusedevel -Dusemallocwrap=no -Dcf_email=srezic@​cpan.org'
hint=recommended
useposix=true
d_sigaction=define
useithreads=undef
usemultiplicity=undef
use64bitint=define
use64bitall=define
uselongdouble=undef
usemymalloc=n
default_inc_excludes_dot=define
bincompat5005=undef
Compiler​:
cc='cc'
ccflags ='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2'
optimize='-O2'
cppflags='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
ccversion=''
gccversion='4.9.2'
gccosandvers=''
intsize=4
longsize=8
ptrsize=8
doublesize=8
byteorder=12345678
doublekind=3
d_longlong=define
longlongsize=8
d_longdbl=define
longdblsize=16
longdblkind=3
ivtype='long'
ivsize=8
nvtype='double'
nvsize=8
Off_t='off_t'
lseeksize=8
alignbytes=8
prototype=define
Linker and Libraries​:
ld='cc'
ldflags =' -fstack-protector-strong -L/usr/local/lib'
libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/4.9/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
libs=-lpthread -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat
perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
libc=libc-2.19.so
so=so
useshrplib=false
libperl=libperl.a
gnulibc_version='2.19'
Dynamic Linking​:
dlsrc=dl_dlopen.xs
dlext=so
d_dlsymun=undef
ccdlflags='-Wl,-E'
cccdlflags='-fPIC'
lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector-strong'

---
@​INC for perl 5.27.9​:
/opt/perl-5.27.9/lib/site_perl/5.27.9/x86_64-linux
/opt/perl-5.27.9/lib/site_perl/5.27.9
/opt/perl-5.27.9/lib/5.27.9/x86_64-linux
/opt/perl-5.27.9/lib/5.27.9

---
Environment for perl 5.27.9​:
HOME=/home/eserte
LANG=en_US.UTF-8
LANGUAGE (unset)
LD_LIBRARY_PATH (unset)
LOGDIR (unset)
PATH=/usr/local/bin​:/usr/bin​:/bin​:/usr/local/sbin​:/usr/sbin​:/sbin​:/home/eserte/bin/linux-gnu​:/home/eserte/bin/sh​:/home/eserte/bin​:/home/eserte/bin/pistachio-perl/bin​:/usr/games​:/home/eserte/devel
PERLDOC=-MPod​::Perldoc​::ToTextOverstrike
PERL_BADLANG (unset)
SHELL=/bin/zsh

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 23, 2018

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 23, 2018

From @eserte

Karl Williamson <public@​khwilliamson.com> writes​:

On 02/23/2018 12​:33 AM, slaven@​rezic.de wrote​:

# New Ticket Created by slaven@​rezic.de
# Please include the string​: [perl #132901]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=132901 >

This is a bug report for perl from slaven@​rezic.de,
generated with the help of perlbug 1.41 running under perl 5.27.9.

-----------------------------------------------------------------
On Linux systems (checked CentOS6 and various Debians) the
following call returns binary garbage instead of the current locale​:

 $ env LC\_ALL=de\_DE\.UTF\-8 perl5\.27\.9 \-MPOSIX=setlocale\,LC\_ALL \-e 'warn setlocale\(LC\_ALL\)'

I don't get garbage, but I'm not getting the shortest output possible
on Ubuntu, which I have a fix for.

Please run the following on a DEBUGGING build, and post the output.
(The problem will not occur on systems that have querylocale(), nor in
locales, such as C, where the radix character is a dot)

env LC_ALL=de_DE.UTF-8 env PERL_DEBUG_LOCALE_INIT=1 perl5.27.9 -DLv
-MPOSIX=setlocale,LC_ALL -e 'warn setlocale(LC_ALL)'

(I don't know the proper syntax of stringing two env's together, so
adjust if necessary)

Attached. This is with a slightly newer perl than 5.27.9 (d3a1131).

Regards,
  Slaven

This does not seem to happen if the current locale is C. And I cannot
reproduce the problem on FreeBSD systems.

The problem does not happen with 5.27.8-157-gef80cd9 or earlier perl
versions.

(Discovered because the test suite of MSCHOUT/Math-Currency-0.52.tar.gz
fails because of this problem).

-----------------------------------------------------------------
---
Flags​:
category=library
severity=low
module=POSIX
---
Site configuration information for perl 5.27.9​:

Configured by eserte at Tue Feb 20 21​:59​:42 CET 2018.

Summary of my perl5 (revision 5 version 27 subversion 9) configuration​:
Platform​:
osname=linux
osvers=3.16.0-4-amd64
archname=x86_64-linux
uname='linux cabulja 3.16.0-4-amd64 #1 smp debian 3.16.51-3 (2017-12-13) x86_64 gnulinux '
config_args='-ds -e -Dprefix=/opt/perl-5.27.9 -Dusedevel -Dusemallocwrap=no -Dcf_email=srezic@​cpan.org'
hint=recommended
useposix=true
d_sigaction=define
useithreads=undef
usemultiplicity=undef
use64bitint=define
use64bitall=define
uselongdouble=undef
usemymalloc=n
default_inc_excludes_dot=define
bincompat5005=undef
Compiler​:
cc='cc'
ccflags ='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2'
optimize='-O2'
cppflags='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
ccversion=''
gccversion='4.9.2'
gccosandvers=''
intsize=4
longsize=8
ptrsize=8
doublesize=8
byteorder=12345678
doublekind=3
d_longlong=define
longlongsize=8
d_longdbl=define
longdblsize=16
longdblkind=3
ivtype='long'
ivsize=8
nvtype='double'
nvsize=8
Off_t='off_t'
lseeksize=8
alignbytes=8
prototype=define
Linker and Libraries​:
ld='cc'
ldflags =' -fstack-protector-strong -L/usr/local/lib'
libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/4.9/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
libs=-lpthread -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat
perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
libc=libc-2.19.so
so=so
useshrplib=false
libperl=libperl.a
gnulibc_version='2.19'
Dynamic Linking​:
dlsrc=dl_dlopen.xs
dlext=so
d_dlsymun=undef
ccdlflags='-Wl,-E'
cccdlflags='-fPIC'
lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector-strong'

---
@​INC for perl 5.27.9​:
/opt/perl-5.27.9/lib/site_perl/5.27.9/x86_64-linux
/opt/perl-5.27.9/lib/site_perl/5.27.9
/opt/perl-5.27.9/lib/5.27.9/x86_64-linux
/opt/perl-5.27.9/lib/5.27.9

---
Environment for perl 5.27.9​:
HOME=/home/eserte
LANG=en_US.UTF-8
LANGUAGE (unset)
LD_LIBRARY_PATH (unset)
LOGDIR (unset)
PATH=/usr/local/bin​:/usr/bin​:/bin​:/usr/local/sbin​:/usr/sbin​:/sbin​:/home/eserte/bin/linux-gnu​:/home/eserte/bin/sh​:/home/eserte/bin​:/home/eserte/bin/pistachio-perl/bin​:/usr/games​:/home/eserte/devel
PERLDOC=-MPod​::Perldoc​::ToTextOverstrike
PERL_BADLANG (unset)
SHELL=/bin/zsh

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 23, 2018

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 23, 2018

From @eserte

--
Slaven Rezic - slaven <at> rezic <dot> de

  Berlin Perl Mongers - http​://berlin.pm.org

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 23, 2018

From @khwilliamson

On 02/23/2018 02​:41 PM, Slaven Rezic wrote​:

Karl Williamson <public@​khwilliamson.com> writes​:

On 02/23/2018 12​:33 AM, slaven@​rezic.de wrote​:

# New Ticket Created by slaven@​rezic.de
# Please include the string​: [perl #132901]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=132901 >

This is a bug report for perl from slaven@​rezic.de,
generated with the help of perlbug 1.41 running under perl 5.27.9.

-----------------------------------------------------------------
On Linux systems (checked CentOS6 and various Debians) the
following call returns binary garbage instead of the current locale​:

  $ env LC\_ALL=de\_DE\.UTF\-8 perl5\.27\.9 \-MPOSIX=setlocale\,LC\_ALL \-e 'warn setlocale\(LC\_ALL\)'

I don't get garbage, but I'm not getting the shortest output possible
on Ubuntu, which I have a fix for.

Please run the following on a DEBUGGING build, and post the output.
(The problem will not occur on systems that have querylocale(), nor in
locales, such as C, where the radix character is a dot)

env LC_ALL=de_DE.UTF-8 env PERL_DEBUG_LOCALE_INIT=1 perl5.27.9 -DLv
-MPOSIX=setlocale,LC_ALL -e 'warn setlocale(LC_ALL)'

(I don't know the proper syntax of stringing two env's together, so
adjust if necessary)

Attached. This is with a slightly newer perl than 5.27.9 (d3a1131).

Unfortunately the logging option you selected was lowercase 'l' instead
of uppercase, and so its the wrong information. You might want to try
instead doing a checkout of smoke-me/khw-locale, which has several bug
fixes related to locales.

Regards,
Slaven

This does not seem to happen if the current locale is C. And I cannot
reproduce the problem on FreeBSD systems.

The problem does not happen with 5.27.8-157-gef80cd9 or earlier perl
versions.

(Discovered because the test suite of MSCHOUT/Math-Currency-0.52.tar.gz
fails because of this problem).

-----------------------------------------------------------------
---
Flags​:
category=library
severity=low
module=POSIX
---
Site configuration information for perl 5.27.9​:

Configured by eserte at Tue Feb 20 21​:59​:42 CET 2018.

Summary of my perl5 (revision 5 version 27 subversion 9) configuration​:
Platform​:
osname=linux
osvers=3.16.0-4-amd64
archname=x86_64-linux
uname='linux cabulja 3.16.0-4-amd64 #1 smp debian 3.16.51-3 (2017-12-13) x86_64 gnulinux '
config_args='-ds -e -Dprefix=/opt/perl-5.27.9 -Dusedevel -Dusemallocwrap=no -Dcf_email=srezic@​cpan.org'
hint=recommended
useposix=true
d_sigaction=define
useithreads=undef
usemultiplicity=undef
use64bitint=define
use64bitall=define
uselongdouble=undef
usemymalloc=n
default_inc_excludes_dot=define
bincompat5005=undef
Compiler​:
cc='cc'
ccflags ='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2'
optimize='-O2'
cppflags='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
ccversion=''
gccversion='4.9.2'
gccosandvers=''
intsize=4
longsize=8
ptrsize=8
doublesize=8
byteorder=12345678
doublekind=3
d_longlong=define
longlongsize=8
d_longdbl=define
longdblsize=16
longdblkind=3
ivtype='long'
ivsize=8
nvtype='double'
nvsize=8
Off_t='off_t'
lseeksize=8
alignbytes=8
prototype=define
Linker and Libraries​:
ld='cc'
ldflags =' -fstack-protector-strong -L/usr/local/lib'
libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/4.9/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
libs=-lpthread -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat
perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
libc=libc-2.19.so
so=so
useshrplib=false
libperl=libperl.a
gnulibc_version='2.19'
Dynamic Linking​:
dlsrc=dl_dlopen.xs
dlext=so
d_dlsymun=undef
ccdlflags='-Wl,-E'
cccdlflags='-fPIC'
lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector-strong'

---
@​INC for perl 5.27.9​:
/opt/perl-5.27.9/lib/site_perl/5.27.9/x86_64-linux
/opt/perl-5.27.9/lib/site_perl/5.27.9
/opt/perl-5.27.9/lib/5.27.9/x86_64-linux
/opt/perl-5.27.9/lib/5.27.9

---
Environment for perl 5.27.9​:
HOME=/home/eserte
LANG=en_US.UTF-8
LANGUAGE (unset)
LD_LIBRARY_PATH (unset)
LOGDIR (unset)
PATH=/usr/local/bin​:/usr/bin​:/bin​:/usr/local/sbin​:/usr/sbin​:/sbin​:/home/eserte/bin/linux-gnu​:/home/eserte/bin/sh​:/home/eserte/bin​:/home/eserte/bin/pistachio-perl/bin​:/usr/games​:/home/eserte/devel
PERLDOC=-MPod​::Perldoc​::ToTextOverstrike
PERL_BADLANG (unset)
SHELL=/bin/zsh

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 24, 2018

From @eserte

Karl Williamson <public@​khwilliamson.com> writes​:

On 02/23/2018 02​:41 PM, Slaven Rezic wrote​:

Karl Williamson <public@​khwilliamson.com> writes​:

On 02/23/2018 12​:33 AM, slaven@​rezic.de wrote​:

# New Ticket Created by slaven@​rezic.de
# Please include the string​: [perl #132901]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=132901 >

This is a bug report for perl from slaven@​rezic.de,
generated with the help of perlbug 1.41 running under perl 5.27.9.

-----------------------------------------------------------------
On Linux systems (checked CentOS6 and various Debians) the
following call returns binary garbage instead of the current locale​:

  $ env LC\_ALL=de\_DE\.UTF\-8 perl5\.27\.9 \-MPOSIX=setlocale\,LC\_ALL \-e 'warn setlocale\(LC\_ALL\)'

I don't get garbage, but I'm not getting the shortest output possible
on Ubuntu, which I have a fix for.

Please run the following on a DEBUGGING build, and post the output.
(The problem will not occur on systems that have querylocale(), nor in
locales, such as C, where the radix character is a dot)

env LC_ALL=de_DE.UTF-8 env PERL_DEBUG_LOCALE_INIT=1 perl5.27.9 -DLv
-MPOSIX=setlocale,LC_ALL -e 'warn setlocale(LC_ALL)'

(I don't know the proper syntax of stringing two env's together, so
adjust if necessary)

Attached. This is with a slightly newer perl than 5.27.9 (d3a1131).

Unfortunately the logging option you selected was lowercase 'l'
instead of uppercase, and so its the wrong information. You might
want to try instead doing a checkout of smoke-me/khw-locale, which has
several bug fixes related to locales.

2nd try, using the smoke-me/khw-locale branch and running

  env LC_ALL=de_DE.UTF-8 PERL_DEBUG_LOCALE_INIT=1 LD_LIBRARY_PATH=$(pwd) ./perl -Ilib -DLv -MPOSIX=setlocale,LC_ALL -e 'warn setlocale(LC_ALL)' >& /tmp/132901-khw-locale.log

Log attached.

Regards,
Slaven

This does not seem to happen if the current locale is C. And I cannot
reproduce the problem on FreeBSD systems.

The problem does not happen with 5.27.8-157-gef80cd9 or earlier perl
versions.

(Discovered because the test suite of MSCHOUT/Math-Currency-0.52.tar.gz
fails because of this problem).

-----------------------------------------------------------------
---
Flags​:
category=library
severity=low
module=POSIX
---
Site configuration information for perl 5.27.9​:

Configured by eserte at Tue Feb 20 21​:59​:42 CET 2018.

Summary of my perl5 (revision 5 version 27 subversion 9) configuration​:
Platform​:
osname=linux
osvers=3.16.0-4-amd64
archname=x86_64-linux
uname='linux cabulja 3.16.0-4-amd64 #1 smp debian 3.16.51-3 (2017-12-13) x86_64 gnulinux '
config_args='-ds -e -Dprefix=/opt/perl-5.27.9 -Dusedevel -Dusemallocwrap=no -Dcf_email=srezic@​cpan.org'
hint=recommended
useposix=true
d_sigaction=define
useithreads=undef
usemultiplicity=undef
use64bitint=define
use64bitall=define
uselongdouble=undef
usemymalloc=n
default_inc_excludes_dot=define
bincompat5005=undef
Compiler​:
cc='cc'
ccflags ='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2'
optimize='-O2'
cppflags='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
ccversion=''
gccversion='4.9.2'
gccosandvers=''
intsize=4
longsize=8
ptrsize=8
doublesize=8
byteorder=12345678
doublekind=3
d_longlong=define
longlongsize=8
d_longdbl=define
longdblsize=16
longdblkind=3
ivtype='long'
ivsize=8
nvtype='double'
nvsize=8
Off_t='off_t'
lseeksize=8
alignbytes=8
prototype=define
Linker and Libraries​:
ld='cc'
ldflags =' -fstack-protector-strong -L/usr/local/lib'
libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/4.9/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
libs=-lpthread -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat
perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
libc=libc-2.19.so
so=so
useshrplib=false
libperl=libperl.a
gnulibc_version='2.19'
Dynamic Linking​:
dlsrc=dl_dlopen.xs
dlext=so
d_dlsymun=undef
ccdlflags='-Wl,-E'
cccdlflags='-fPIC'
lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector-strong'

---
@​INC for perl 5.27.9​:
/opt/perl-5.27.9/lib/site_perl/5.27.9/x86_64-linux
/opt/perl-5.27.9/lib/site_perl/5.27.9
/opt/perl-5.27.9/lib/5.27.9/x86_64-linux
/opt/perl-5.27.9/lib/5.27.9

---
Environment for perl 5.27.9​:
HOME=/home/eserte
LANG=en_US.UTF-8
LANGUAGE (unset)
LD_LIBRARY_PATH (unset)
LOGDIR (unset)
PATH=/usr/local/bin​:/usr/bin​:/bin​:/usr/local/sbin​:/usr/sbin​:/sbin​:/home/eserte/bin/linux-gnu​:/home/eserte/bin/sh​:/home/eserte/bin​:/home/eserte/bin/pistachio-perl/bin​:/usr/games​:/home/eserte/devel
PERLDOC=-MPod​::Perldoc​::ToTextOverstrike
PERL_BADLANG (unset)
SHELL=/bin/zsh

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 24, 2018

From @eserte

132901-khw-locale.log

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 24, 2018

From @eserte

--
Slaven Rezic - slaven <at> rezic <dot> de

  Berlin Perl Mongers - http​://berlin.pm.org

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 24, 2018

From @khwilliamson

On 02/24/2018 02​:00 AM, Slaven Rezic wrote​:

Karl Williamson <public@​khwilliamson.com> writes​:

On 02/23/2018 02​:41 PM, Slaven Rezic wrote​:

Karl Williamson <public@​khwilliamson.com> writes​:

On 02/23/2018 12​:33 AM, slaven@​rezic.de wrote​:

# New Ticket Created by slaven@​rezic.de
# Please include the string​: [perl #132901]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=132901 >

This is a bug report for perl from slaven@​rezic.de,
generated with the help of perlbug 1.41 running under perl 5.27.9.

-----------------------------------------------------------------
On Linux systems (checked CentOS6 and various Debians) the
following call returns binary garbage instead of the current locale​:

   $ env LC\_ALL=de\_DE\.UTF\-8 perl5\.27\.9 \-MPOSIX=setlocale\,LC\_ALL \-e 'warn setlocale\(LC\_ALL\)'

I don't get garbage, but I'm not getting the shortest output possible
on Ubuntu, which I have a fix for.

Please run the following on a DEBUGGING build, and post the output.
(The problem will not occur on systems that have querylocale(), nor in
locales, such as C, where the radix character is a dot)

env LC_ALL=de_DE.UTF-8 env PERL_DEBUG_LOCALE_INIT=1 perl5.27.9 -DLv
-MPOSIX=setlocale,LC_ALL -e 'warn setlocale(LC_ALL)'

(I don't know the proper syntax of stringing two env's together, so
adjust if necessary)

Attached. This is with a slightly newer perl than 5.27.9 (d3a1131).

Unfortunately the logging option you selected was lowercase 'l'
instead of uppercase, and so its the wrong information. You might
want to try instead doing a checkout of smoke-me/khw-locale, which has
several bug fixes related to locales.

2nd try, using the smoke-me/khw-locale branch and running

 env LC\_ALL=de\_DE\.UTF\-8 PERL\_DEBUG\_LOCALE\_INIT=1 LD\_LIBRARY\_PATH=$\(pwd\) \./perl \-Ilib \-DLv \-MPOSIX=setlocale\,LC\_ALL \-e 'warn setlocale\(LC\_ALL\)' >& /tmp/132901\-khw\-locale\.log

Log attached.

Please send me config.h for this system

Regards,
Slaven

This does not seem to happen if the current locale is C. And I cannot
reproduce the problem on FreeBSD systems.

The problem does not happen with 5.27.8-157-gef80cd9 or earlier perl
versions.

(Discovered because the test suite of MSCHOUT/Math-Currency-0.52.tar.gz
fails because of this problem).

-----------------------------------------------------------------
---
Flags​:
category=library
severity=low
module=POSIX
---

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 25, 2018

From @khwilliamson

On 02/24/2018 01​:14 PM, Karl Williamson wrote​:

On 02/24/2018 02​:00 AM, Slaven Rezic wrote​:

Karl Williamson <public@​khwilliamson.com> writes​:

On 02/23/2018 02​:41 PM, Slaven Rezic wrote​:

Karl Williamson <public@​khwilliamson.com> writes​:

On 02/23/2018 12​:33 AM, slaven@​rezic.de wrote​:

# New Ticket Created by  slaven@​rezic.de
# Please include the string​:  [perl #132901]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=132901 >

This is a bug report for perl from slaven@​rezic.de,
generated with the help of perlbug 1.41 running under perl 5.27.9.

I believe this is fixed in blead; please verify.

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 25, 2018

From @eserte

"karl williamson via RT" <perlbug-followup@​perl.org> writes​:

On 02/24/2018 01​:14 PM, Karl Williamson wrote​:

On 02/24/2018 02​:00 AM, Slaven Rezic wrote​:

Karl Williamson <public@​khwilliamson.com> writes​:

On 02/23/2018 02​:41 PM, Slaven Rezic wrote​:

Karl Williamson <public@​khwilliamson.com> writes​:

On 02/23/2018 12​:33 AM, slaven@​rezic.de wrote​:

# New Ticket Created by  slaven@​rezic.de
# Please include the string​:  [perl #132901]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=132901 >

This is a bug report for perl from slaven@​rezic.de,
generated with the help of perlbug 1.41 running under perl 5.27.9.

I believe this is fixed in blead; please verify.

No, it's still not fixed​:

$ env LC_ALL=de_DE.UTF-8 ./perl -Ilib -MTest​::More=no_plan -MPOSIX=setlocale,LC_ALL -e 'is setlocale(LC_ALL), "de_DE.UTF-8"'
not ok 1
# Failed test at -e line 1.
# got​: '�^'
# expected​: 'de_DE.UTF-8'
1..1
# Looks like you failed 1 test of 1.

$ env LC_ALL=de_DE.UTF-8 ./perl -Ilib -V|grep Commit.id
  Commit id​: c1cc29f

--
Slaven Rezic - slaven <at> rezic <dot> de
  BBBike - route planner for cyclists in Berlin
  WWW version​: http​://www.bbbike.de
  Perl/Tk version for Unix and Windows​: http​://bbbike.sourceforge.net

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 25, 2018

From @khwilliamson

On 02/25/2018 03​:03 AM, Slaven Rezic wrote​:

"karl williamson via RT" <perlbug-followup@​perl.org> writes​:

On 02/24/2018 01​:14 PM, Karl Williamson wrote​:

On 02/24/2018 02​:00 AM, Slaven Rezic wrote​:

Karl Williamson <public@​khwilliamson.com> writes​:

On 02/23/2018 02​:41 PM, Slaven Rezic wrote​:

Karl Williamson <public@​khwilliamson.com> writes​:

On 02/23/2018 12​:33 AM, slaven@​rezic.de wrote​:

# New Ticket Created by  slaven@​rezic.de
# Please include the string​:  [perl #132901]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=132901 >

This is a bug report for perl from slaven@​rezic.de,
generated with the help of perlbug 1.41 running under perl 5.27.9.

I believe this is fixed in blead; please verify.

No, it's still not fixed​:

$ env LC_ALL=de_DE.UTF-8 ./perl -Ilib -MTest​::More=no_plan -MPOSIX=setlocale,LC_ALL -e 'is setlocale(LC_ALL), "de_DE.UTF-8"'
not ok 1
# Failed test at -e line 1.
# got​: '�^'
# expected​: 'de_DE.UTF-8'
1..1
# Looks like you failed 1 test of 1.

$ env LC_ALL=de_DE.UTF-8 ./perl -Ilib -V|grep Commit.id
Commit id​: c1cc29f

Sorry, somehow not all the commits I intended to push actually got
pushed. The commit I believe will fix the problem is
7a60910d1bd9081951d3509805c035b0c4bcbe79

And, yes, I will add a test

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 25, 2018

From @eserte

Karl Williamson <public@​khwilliamson.com> writes​:

On 02/25/2018 03​:03 AM, Slaven Rezic wrote​:

"karl williamson via RT" <perlbug-followup@​perl.org> writes​:

On 02/24/2018 01​:14 PM, Karl Williamson wrote​:

On 02/24/2018 02​:00 AM, Slaven Rezic wrote​:

Karl Williamson <public@​khwilliamson.com> writes​:

On 02/23/2018 02​:41 PM, Slaven Rezic wrote​:

Karl Williamson <public@​khwilliamson.com> writes​:

On 02/23/2018 12​:33 AM, slaven@​rezic.de wrote​:

# New Ticket Created by  slaven@​rezic.de
# Please include the string​:  [perl #132901]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=132901 >

This is a bug report for perl from slaven@​rezic.de,
generated with the help of perlbug 1.41 running under perl 5.27.9.

I believe this is fixed in blead; please verify.

No, it's still not fixed​:

$ env LC_ALL=de_DE.UTF-8 ./perl -Ilib -MTest​::More=no_plan -MPOSIX=setlocale,LC_ALL -e 'is setlocale(LC_ALL), "de_DE.UTF-8"'
not ok 1
# Failed test at -e line 1.
# got​: '�^'
# expected​: 'de_DE.UTF-8'
1..1
# Looks like you failed 1 test of 1.

$ env LC_ALL=de_DE.UTF-8 ./perl -Ilib -V|grep Commit.id
Commit id​: c1cc29f

Sorry, somehow not all the commits I intended to push actually got
pushed. The commit I believe will fix the problem is
7a60910d1bd9081951d3509805c035b0c4bcbe79

It's still not pushed​:

$ git fetch && git checkout 7a60910d1bd9081951d3509805c035b0c4bcbe79
fatal​: reference is not a tree​: 7a60910d1bd9081951d3509805c035b0c4bcbe79

And, yes, I will add a test

--
Slaven Rezic - slaven <at> rezic <dot> de

  Berlin Perl Mongers - http​://berlin.pm.org

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 25, 2018

From @khwilliamson

On 02/25/2018 09​:26 AM, Slaven Rezic wrote​:

Karl Williamson <public@​khwilliamson.com> writes​:

On 02/25/2018 03​:03 AM, Slaven Rezic wrote​:

"karl williamson via RT" <perlbug-followup@​perl.org> writes​:

On 02/24/2018 01​:14 PM, Karl Williamson wrote​:

On 02/24/2018 02​:00 AM, Slaven Rezic wrote​:

Karl Williamson <public@​khwilliamson.com> writes​:

On 02/23/2018 02​:41 PM, Slaven Rezic wrote​:

Karl Williamson <public@​khwilliamson.com> writes​:

On 02/23/2018 12​:33 AM, slaven@​rezic.de wrote​:

# New Ticket Created by  slaven@​rezic.de
# Please include the string​:  [perl #132901]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=132901 >

This is a bug report for perl from slaven@​rezic.de,
generated with the help of perlbug 1.41 running under perl 5.27.9.

I believe this is fixed in blead; please verify.

No, it's still not fixed​:

$ env LC_ALL=de_DE.UTF-8 ./perl -Ilib -MTest​::More=no_plan -MPOSIX=setlocale,LC_ALL -e 'is setlocale(LC_ALL), "de_DE.UTF-8"'
not ok 1
# Failed test at -e line 1.
# got​: '�^'
# expected​: 'de_DE.UTF-8'
1..1
# Looks like you failed 1 test of 1.

$ env LC_ALL=de_DE.UTF-8 ./perl -Ilib -V|grep Commit.id
Commit id​: c1cc29f

Sorry, somehow not all the commits I intended to push actually got
pushed. The commit I believe will fix the problem is
7a60910d1bd9081951d3509805c035b0c4bcbe79

It's still not pushed​:

$ git fetch && git checkout 7a60910d1bd9081951d3509805c035b0c4bcbe79
fatal​: reference is not a tree​: 7a60910d1bd9081951d3509805c035b0c4bcbe79

Embarrassingly, I guess the push failed, and I didn't look carefully.
Now, it definitely is, as
33e5a35

And, yes, I will add a test

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 25, 2018

From @eserte

"karl williamson via RT" <perlbug-followup@​perl.org> writes​:

On 02/25/2018 09​:26 AM, Slaven Rezic wrote​:

Karl Williamson <public@​khwilliamson.com> writes​:

On 02/25/2018 03​:03 AM, Slaven Rezic wrote​:

"karl williamson via RT" <perlbug-followup@​perl.org> writes​:

On 02/24/2018 01​:14 PM, Karl Williamson wrote​:

On 02/24/2018 02​:00 AM, Slaven Rezic wrote​:

Karl Williamson <public@​khwilliamson.com> writes​:

On 02/23/2018 02​:41 PM, Slaven Rezic wrote​:

Karl Williamson <public@​khwilliamson.com> writes​:

On 02/23/2018 12​:33 AM, slaven@​rezic.de wrote​:

# New Ticket Created by  slaven@​rezic.de
# Please include the string​:  [perl #132901]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=132901 >

This is a bug report for perl from slaven@​rezic.de,
generated with the help of perlbug 1.41 running under perl 5.27.9.

I believe this is fixed in blead; please verify.

No, it's still not fixed​:

$ env LC_ALL=de_DE.UTF-8 ./perl -Ilib -MTest​::More=no_plan -MPOSIX=setlocale,LC_ALL -e 'is setlocale(LC_ALL), "de_DE.UTF-8"'
not ok 1
# Failed test at -e line 1.
# got​: '�^'
# expected​: 'de_DE.UTF-8'
1..1
# Looks like you failed 1 test of 1.

$ env LC_ALL=de_DE.UTF-8 ./perl -Ilib -V|grep Commit.id
Commit id​: c1cc29f

Sorry, somehow not all the commits I intended to push actually got
pushed. The commit I believe will fix the problem is
7a60910d1bd9081951d3509805c035b0c4bcbe79

It's still not pushed​:

$ git fetch && git checkout 7a60910d1bd9081951d3509805c035b0c4bcbe79
fatal​: reference is not a tree​: 7a60910d1bd9081951d3509805c035b0c4bcbe79

Embarrassingly, I guess the push failed, and I didn't look carefully.
Now, it definitely is, as
33e5a35

Yes, the commit is there, and the oneliner above passes now.

Thanks & regards,
  Slaven

--
Slaven Rezic - slaven <at> rezic <dot> de

  Berlin Perl Mongers - http​://berlin.pm.org

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 25, 2018

From @khwilliamson

On 02/25/2018 10​:21 AM, Slaven Rezic wrote​:

$ env LC_ALL=de_DE.UTF-8 ./perl -Ilib -MTest​::More=no_plan -MPOSIX=setlocale,LC_ALL -e 'is setlocale(LC_ALL), "de_DE.UTF-8"'
not ok 1
# Failed test at -e line 1.
# got​: '�^'
# expected​: 'de_DE.UTF-8'
1..1
# Looks like you failed 1 test of 1.

I realize now in looking at the tests why there is not already a test
for this. The reason is that the return from setlocale is explictly
documented to be opaque; not necessarily human readable. It is required
only that a future setlocale() changing the locale to the value returned
properly work.

That said, I know of no system that actually doesn't return a human
readable name, but it may not exactly match the input one. It might be
missing (or gained) a hyphen, or the casing may be different. And where
individual categories aren't all the same, LC_ALL is represented
differently on different systems. (On Linux, it is a list of the
individual categories and their values with each component separated by
a semi-colon).

I suppose a test could be written which strips off the hyphens and
lowercases everything, and then do the comparison, and then we could
skip or adjust this test if any platforms showed up if failed for.

I'm open to hearing ideas.

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 25, 2018

From @cpansprout

On Sun, 25 Feb 2018 11​:47​:40 -0800, public@​khwilliamson.com wrote​:

On 02/25/2018 10​:21 AM, Slaven Rezic wrote​:

$ env LC_ALL=de_DE.UTF-8 ./perl -Ilib -MTest​::More=no_plan
-MPOSIX=setlocale,LC_ALL -e 'is setlocale(LC_ALL), "de_DE.UTF-8"'
not ok 1
# Failed test at -e line 1.
# got​: '�^'
# expected​: 'de_DE.UTF-8'
1..1
# Looks like you failed 1 test of 1.

I realize now in looking at the tests why there is not already a test
for this. The reason is that the return from setlocale is explictly
documented to be opaque; not necessarily human readable. It is
required
only that a future setlocale() changing the locale to the value
returned
properly work.

That said, I know of no system that actually doesn't return a human
readable name, but it may not exactly match the input one. It might
be
missing (or gained) a hyphen, or the casing may be different. And
where
individual categories aren't all the same, LC_ALL is represented
differently on different systems. (On Linux, it is a list of the
individual categories and their values with each component separated
by
a semi-colon).

I suppose a test could be written which strips off the hyphens and
lowercases everything, and then do the comparison, and then we could
skip or adjust this test if any platforms showed up if failed for.

I'm open to hearing ideas.

Tests that need constant tweaking are the maintainer’s bane. Could we do some kind of ‘loose match’ that requires at least, say, four or five of the same ASCII alphanumerical characters to be present in both strings, and in the right order?

If you transform de_DE.UTF-8 to /d.*e.*d.*e.*/aai this might actually work and require little tweaking in the long term. It would certainly have caught the bug.

This would work​:

$regexp = substr fc($locale) =~ y/a-z0-9//cdr, 0, 4, =~ s/./$&amp;.*/gr;

--

Father Chrysostomos

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 25, 2018

From @khwilliamson

OP confirms that is fixed by 33e5a35
--
Karl Williamson

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 25, 2018

@khwilliamson - Status changed from 'open' to 'resolved'

@p5pRT p5pRT closed this as completed Feb 25, 2018
@p5pRT
Copy link
Author

@p5pRT p5pRT commented Mar 30, 2018

From zefram@fysh.org

Karl Williamson wrote​:

                                          It is required only that a

future setlocale() changing the locale to the value returned properly work.

That's obviously the correct way to test it​: feed the value to setlocale()
and check that locale-dependent things behave in accordance with the
expected locale.

-zefram

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant