perl-5.18.0 apparently miscalculates the NV 1e-298

[p5pRT]

Migrated from rt.perl.org#120363 (status was 'open')

Searchable as RT120363$

[p5pRT]

From @sisyphus

Hi,
On ubuntu-12.04LTS I have 2 builds of perl-5.18.0, both built by gcc-4.6.3.

One of those perls has nvtype of 'double', the other has nvtype of 'long
double' (64-bit significand).

I run the following Inline​::C script​:

#######################

#!perl -l
use warnings;
use strict;
use Config;

#use Inline C => Config =>
# USING => 'ParseRegExp',
# BUILD_NOISY => 1;

use Inline C => <<'EOC';

SV * get_problem_val(long nvsize) {
  if(nvsize == 8) return newSVnv(1e-298);
  return newSVnv(1e-298L);
}

EOC

my $s = $Config{nvsize};
print "\n\$Config{nvtype}​: $Config{nvtype}";
print "Different values" if 1e-298 != get_problem_val($s);

if($Config{nvsize} != 8) {
print scalar reverse unpack "b64", pack "D", 1e-298;
print scalar reverse unpack "b64", pack "D", get_problem_val($s);
}
else {
print scalar reverse unpack "b53", pack "F", 1e-298;
print scalar reverse unpack "b53", pack "F", get_problem_val($s);
};
}

#######################

When I run that on the "double" build of perl-5.18.0, it produces the
following output​:

$Config{nvtype}​: double
Different values
10000101111100000100011010000010100100111111000011011
10000101111100000100011010000010100100111111000011101

When I run that on the "long double" build of perl-5.18.0, it produces​:

$Config{nvtype}​: long double
Different values
1000010111110000010001101000001010010011111100001110101101010000
1000010111110000010001101000001010010011111100001110101101001110

In both instances it's the second of the 2 values (ie the value returned
from C, not the value that perl assigns) that's correct.

I presume there would be other values that also produce the discrepancy,
but I haven't sought any out.

Here are the 'perl -V' outputs for both perls - firstly, the "double" build​:

########################

This is perl 5, version 18, subversion 0 (v5.18.0) built for
x86_64-linux-thread-multi

Copyright 1987-2013, Larry Wall

Perl may be copied only under the terms of either the Artistic License
or the
GNU General Public License, which may be found in the Perl 5 source kit.

Complete documentation for Perl, including FAQ lists, should be found on
this system using "man perl" or "perldoc perl". If you have access to the
Internet, point your browser at http​://www.perl.org/, the Perl Home Page.

sisyphus@​sisyphus5-desktop​:~/pscrpt/inline$ perl -V
Summary of my perl5 (revision 5 version 18 subversion 0) configuration​:

  Platform​:
  osname=linux, osvers=3.5.0-34-generic,
archname=x86_64-linux-thread-multi
  uname='linux sisyphus5-desktop 3.5.0-34-generic #55~precise1-ubuntu
smp fri jun 7 16​:25​:50 utc 2013 x86_64 x86_64 x86_64 gnulinux '
  config_args='-des -Duse64bitint -Duse64bitall -Dusethreads
-Dprefix=/home/sisyphus/perl518-multi'
  hint=recommended, useposix=true, d_sigaction=define
  useithreads=define, usemultiplicity=define
  useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
  use64bitint=define, use64bitall=define, uselongdouble=undef
  usemymalloc=n, bincompat5005=undef
  Compiler​:
  cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing
-pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64',
  optimize='-O2',
  cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe
-fstack-protector -I/usr/local/include'
  ccversion='', gccversion='4.6.3', gccosandvers=''
  intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
  d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
  ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
  alignbytes=8, prototype=define
  Linker and Libraries​:
  ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
  libpth=/usr/local/lib /lib/x86_64-linux-gnu /lib/../lib
/usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib /usr/lib
  libs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
  perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
  libc=, so=so, useshrplib=false, libperl=libperl.a
  gnulibc_version='2.15'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
  cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib
-fstack-protector'

Characteristics of this binary (from libperl)​:
  Compile-time options​: HAS_TIMES MULTIPLICITY PERLIO_LAYERS
  PERL_DONT_CREATE_GVSV
  PERL_HASH_FUNC_ONE_AT_A_TIME_HARD
  PERL_IMPLICIT_CONTEXT PERL_MALLOC_WRAP
  PERL_PRESERVE_IVUV PERL_SAWAMPERSAND USE_64_BIT_ALL
  USE_64_BIT_INT USE_ITHREADS USE_LARGE_FILES
  USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE
  USE_LOCALE_NUMERIC USE_PERLIO USE_PERL_ATOF
  USE_REENTRANT_API
  Built under linux
  Compiled at Jul 5 2013 19​:03​:19
  @​INC​:
/home/sisyphus/perl518-multi/lib/site_perl/5.18.0/x86_64-linux-thread-multi
  /home/sisyphus/perl518-multi/lib/site_perl/5.18.0
/home/sisyphus/perl518-multi/lib/5.18.0/x86_64-linux-thread-multi
  /home/sisyphus/perl518-multi/lib/5.18.0
  .

########################

and the "long double" build​:

########################
This is perl 5, version 18, subversion 0 (v5.18.0) built for x86_64-linux-ld

Copyright 1987-2013, Larry Wall

Perl may be copied only under the terms of either the Artistic License
or the
GNU General Public License, which may be found in the Perl 5 source kit.

Complete documentation for Perl, including FAQ lists, should be found on
this system using "man perl" or "perldoc perl". If you have access to the
Internet, point your browser at http​://www.perl.org/, the Perl Home Page.

sisyphus@​sisyphus5-desktop​:~/pscrpt/inline$ perl -V
Summary of my perl5 (revision 5 version 18 subversion 0) configuration​:

  Platform​:
  osname=linux, osvers=3.5.0-23-generic, archname=x86_64-linux-ld
  uname='linux sisyphus5-desktop 3.5.0-23-generic #35~precise1-ubuntu
smp fri jan 25 17​:13​:26 utc 2013 x86_64 x86_64 x86_64 gnulinux '
  config_args='-des -Duse64bitint -Duse64bitall -Duselongdouble
-Dprefix=/home/sisyphus/perl518'
  hint=recommended, useposix=true, d_sigaction=define
  useithreads=undef, usemultiplicity=undef
  useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
  use64bitint=define, use64bitall=define, uselongdouble=define
  usemymalloc=n, bincompat5005=undef
  Compiler​:
  cc='cc', ccflags ='-fno-strict-aliasing -pipe -fstack-protector
-I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
  optimize='-O2',
  cppflags='-fno-strict-aliasing -pipe -fstack-protector
-I/usr/local/include'
  ccversion='', gccversion='4.6.3', gccosandvers=''
  intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
  d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
  ivtype='long', ivsize=8, nvtype='long double', nvsize=16,
Off_t='off_t', lseeksize=8
  alignbytes=16, prototype=define
  Linker and Libraries​:
  ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
  libpth=/usr/local/lib /lib/x86_64-linux-gnu /lib/../lib
/usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib /usr/lib
  libs=-lnsl -ldl -lm -lcrypt -lutil -lc
  perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc
  libc=, so=so, useshrplib=false, libperl=libperl.a
  gnulibc_version='2.15'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
  cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib
-fstack-protector'

Characteristics of this binary (from libperl)​:
  Compile-time options​: HAS_TIMES PERLIO_LAYERS PERL_DONT_CREATE_GVSV
  PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_MALLOC_WRAP
  PERL_PRESERVE_IVUV PERL_SAWAMPERSAND USE_64_BIT_ALL
  USE_64_BIT_INT USE_LARGE_FILES USE_LOCALE
  USE_LOCALE_COLLATE USE_LOCALE_CTYPE
  USE_LOCALE_NUMERIC USE_LONG_DOUBLE USE_PERLIO
  USE_PERL_ATOF
  Built under linux
  Compiled at Jul 3 2013 20​:20​:11
  @​INC​:
  /home/sisyphus/perl518/lib/site_perl/5.18.0/x86_64-linux-ld
  /home/sisyphus/perl518/lib/site_perl/5.18.0
  /home/sisyphus/perl518/lib/5.18.0/x86_64-linux-ld
  /home/sisyphus/perl518/lib/5.18.0
  .

########################

Cheers,
Rob

[p5pRT]

From zefram@fysh.org

Sisyphus wrote​:

I presume there would be other values that also produce the discrepancy,

See [perl #41202], where Perl gets the wrong value and libc gets the
right value for a text->float conversion where the value represented
textually can be exactly represented as an NV.

-zefram

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From @sisyphus

-----Original Message-----
From​: Zefram via RT
Sent​: Friday, October 25, 2013 10​:17 PM
To​: sisyphus1@​optusnet.com.au
Subject​: Re​: [perl #120363] perl-5.18.0 apparently miscalculates the NV
1e-298

Sisyphus wrote​:

I presume there would be other values that also produce the discrepancy,

See [perl #41202], where Perl gets the wrong value and libc gets the
right value for a text->float conversion where the value represented
textually can be exactly represented as an NV.

Is there a connection ?
That looks quite different - I'm not doing any text-float conversion and
perl represents the value 2^70 perfectly accurately​:

Where nvtype is "double"​:
$ perl -le 'print scalar reverse unpack "b53", pack "F", 2 ** 70;'
10000000000000000000000000000000000000000000000000000

And where nvtype is "long double"
$ perl -le 'print scalar reverse unpack "b64", pack "D", 2 ** 70;'
1000000000000000000000000000000000000000000000000000000000000000

(Sorry if I've missed the point.)

Cheers,
Rob

[p5pRT]

From zefram@fysh.org

sisyphus1@​optusnet.com.au wrote​:

Is there a connection ?

They're the same class of problem​: libc performs better than Perl at
text->float conversion.

That looks quite different - I'm not doing any text-float conversion

You gave Perl the text "1e-298" in a context where it was treated as a
floating-point literal, thus calling upon Perl to convert it to a floating
point value, and you looked at the resulting value. In the same manner,
in [perl #41202] I gave Perl the text "1180591620717411303424.0" as a
floating-point literal. Same data flow in both cases, and the crucial
operation is a conversion from decimal text to floating point.

and perl represents the value 2^70 perfectly accurately​:

That's the difference between these two tickets. 2**70 can be represented
exactly as an NV, but 10**-298 cannot. By using an exactly-representable
value, [perl #41202] illustrates that the issue is not merely the
unavoidable rounding that comes from converting a value that can't be
represented exactly. (The fact that the differences you see with "1e-298"
are of 2 ulp also illustrate that the issue isn't just this rounding.
A single rounding operation being handled differently in different places
could account for 1 ulp.)

Oh, another way to see the "1e-298" difference​: compare "1e-298" against
"10**-298". The latter (computed in Perl) gives me the same result that
you got from the C compiler for "1e-298". Presumably the floating-point
pow() operation is giving a correctly-rounded result.

-zefram

[p5pRT]

From @sisyphus

-----Original Message-----
From​: Zefram via RT
Sent​: Friday, October 25, 2013 11​:10 PM
To​: sisyphus1@​optusnet.com.au
Subject​: Re​: [perl #120363] perl-5.18.0 apparently miscalculates the NV
1e-298

sisyphus1@​optusnet.com.au wrote​:

Is there a connection ?

They're the same class of problem​: libc performs better than Perl at
text->float conversion.

That looks quite different - I'm not doing any text-float conversion

You gave Perl the text "1e-298" in a context where it was treated as a
floating-point literal, thus calling upon Perl to convert it to a floating
point value, and you looked at the resulting value. In the same manner,
in [perl #41202] I gave Perl the text "1180591620717411303424.0" as a
floating-point literal. Same data flow in both cases, and the crucial
operation is a conversion from decimal text to floating point.

Ok ... and this has been known about for quite some time ... which leads to
me ponder that it might be a while before it gets fixed ?

[snip]

Oh, another way to see the "1e-298" difference​: compare "1e-298" against
"10**-298". The latter (computed in Perl) gives me the same result that
you got from the C compiler for "1e-298". Presumably the floating-point
pow() operation is giving a correctly-rounded result.

Yes, on the perl-5.18.0 built with nvtype eq 'double', 10 ** -298 produces
the C compiler value for me, too.
But on the perl built with nvtype eq 'long double', things are worse with 10
** -298​:

$ perl -le 'print scalar reverse unpack "b64", pack "D", 10 ** -298;'
1000010111110000010001101000001010010011111100001110101111000011

That differs from the C value for 1e-298 beginning at the 57th bit.

Cheers,
Rob

[p5pRT]

From zefram@fysh.org

sisyphus1@​optusnet.com.au wrote​:

But on the perl built with nvtype eq 'long double', things are worse
with 10 ** -298​:

Ooh, interesting. OK, I get the same values as you. In summary (via
Data​::Float​::float_hex())​:

+0x1.0be08d0527e1d69cp-990 C source (gcc) 1e-298L, C strtold "1e-298",
  C powl(10.0L, -298.0L),
  actual closest representable to 10**-298
+0x1.0be08d0527e1d6a0p-990 Perl (source or runtime string coercion) 1e-298
+0x1.0be08d0527e1d786p-990 Perl 10**-298

I'm mystified as to how Perl 10**-298 gets a different result from C
powl(), because pp_pow() clearly uses powl() underneath. Even if it
were using the lower precision pow(), that would lead to the low-order
bits being zero, not the inaccurate bits that we see. Anyway, I reckon
pp_pow giving the wrong answer is a separate bug from the text->float
conversion errors that this ticket started with.

-zefram

[p5pRT]

From zefram@fysh.org

I wrote​:

I'm mystified as to how Perl 10**-298 gets a different result from C
powl(), because pp_pow() clearly uses powl() underneath.

I've narrowed this difference down to C constant folding. powl() on
literal arguments gets folded at C compile time, and produces a different
result from powl() that actually runs at runtime. Not a Perl bug.
(Still a separate bug from the text->float conversion errors that this
ticket is really concerned with.)

-zefram

[p5pRT]

From @sisyphus

-----Original Message-----
From​: Zefram via RT
Sent​: Monday, October 28, 2013 2​:32 AM
To​: sisyphus1@​optusnet.com.au
Subject​: Re​: [perl #120363] perl-5.18.0 apparently miscalculates the NV
1e-298

I wrote​:

I'm mystified as to how Perl 10**-298 gets a different result from C
powl(), because pp_pow() clearly uses powl() underneath.

I've narrowed this difference down to C constant folding. powl() on
literal arguments gets folded at C compile time, and produces a different
result from powl() that actually runs at runtime. Not a Perl bug.
(Still a separate bug from the text->float conversion errors that this
ticket is really concerned with.)

Oh, yes - I can see this.
It's the difference between doing​:

long double ld;
ld = powl(10.0L, -298.0L);

and​:

long double ld, val, pow;
val= 10.0L;
pow = -298.0L;
ld = powl(val, pow);

And this would be a bug in the constant folding, I presume.
On Windows, I see the same bug in (mingw64 ports of) both gcc-4.6.3 and
gcc-4.8.1.

Cheers,
Rob

[p5pRT]

From zefram@fysh.org

sisyphus1@​optusnet.com.au wrote​:

And this would be a bug in the constant folding, I presume.

Well, it's a bug that the constant folding doesn't match the runtime
behaviour, but the constant folding was more accurate. So the main bug
is in libc, having an inaccurate powl().

I reported it as a libc bug via Debian
<http​://bugs.debian.org/cgi-bin/bugreport.cgi?bug=727841>, and it turned
out it's been fixed already. glibc 2.13 has the inaccurate powl(); in
glibc 2.17 powl() produces the same (correct) result as gcc's constant
folding.

-zefram

[richardleach]

Is there still a bug here? There seemed to be two different situations:

1. A bug in libc, reported by Zefram and fixed over a decade ago
2. A difference in values coming from perl vs C

For the latter, I ran the print scalar reverse unpack "b53", pack "F", 1e-298; test case on the $Config{nvtype}​: double build from a recent blead that I have handy. It produced the "correct" value as identified earlier in the thread. That is to say, the originally reported output was:

10000101111100000100011010000010100100111111000011011 [from perl, incorrect]
10000101111100000100011010000010100100111111000011101 [from C, correct]

I got this from blead today:

10000101111100000100011010000010100100111111000011101

[sisyphus]

I've just checked, and there's still an issue with 10 ** -298 on (Windows only) long double builds - but that will be a bug in powl(), not a perl bug.
If I haven't previously done it, I'll raise this Windows powl() issue with the appropriate body (either mingw-w64 or gcc).

For both 'double' and 'long double' builds of perl, I found no issue with the assignment of 1e-298 on either Windows (perl-5.41.6) or Ubuntu (perl-5.41.4) - so this ticket is closable, IMO.
I don't have permission to close this github issue. I guess that's related to it having been migrated from rt.

[sisyphus]

there's still an issue with 10 ** -298 on (Windows only) long double builds

Actually there's also an issue for other nvtypes (double and __float128), and not just for Windows.
I find the same type of issue on Ubuntu-24.04 with perl-5.40.0 - though the actual values that fail can vary between different perl versions and different systems.

I haven't found a perl on either Windows or Ubuntu where the following script passes all (620) tests:

use strict;
use warnings;
use Config;
use Test::More;

for my $p(1..310) {
  cmp_ok("1e-$p", '==', 10 ** -$p, "10 ** -$p ok");
  cmp_ok("1e$p" , '==', 10 **  $p, "10 **  $p ok");
}

done_testing();

I'd be interested to hear if anyone has a perl on which that script does pass all tests.

The bug is in the underlying math library, so it's not a perl bug IMO.