floating-point binary-decimal conversion bugs

[p5pRT]

Migrated from rt.perl.org#47602 (status was 'open')

Searchable as RT47602$

[p5pRT]

From vincent-perl@vinc17.net

Created by vincent@vinc17.org

The following Perl script gives strange results on most machines
(see the output below).

#!/usr/bin/env perl

use strict;
use Config;

my $uld = $Config{uselongdouble} ? ', uselongdouble' : '';
print "Perl $], $Config{osname} $Config{osvers}, $Config{archname}\n",
  (join ', ', map "$_=$Config{$_}", qw/nvtype nvsize d_longdbl longdblsize/),
  "$uld\nlibc=$Config{libc}, libperl=$Config{libperl}\n";

my $x = 7.0868766365730135e-268;
print unpack("H16", pack("d", $x)), "\n";
while ($x < 4503599627370496.0)
  { $x *= 2.0 }
printf "Significand​: %.8f\n", $x;
printf "Significand​: %.0f\n", $x;

__END__

Below, only vin, zaurus and allo-psmn are correct. I first thought
this was a bug in the C library, but the strtod() function is correct
on "7.0868766365730135e-268" on a few machines I've tested. Note that
if double precision is used internally, then $x has an integral value
after the loop (4503599627370496 is 2^52).

* vin.lip.ens-lyon.fr
Perl 5.008008, linux 2.6.22.10, i486-linux-gnu-thread-multi
nvtype=double, nvsize=8, d_longdbl=define, longdblsize=12
libc=/lib/libc-2.6.1.so, libperl=libperl.so.5.8.8
1122334455667708
Significand​: 6586440866800145.00000000
Significand​: 6586440866800145

* courge.ens-lyon.fr
Perl 5.008008, linux 2.6.22-3-amd64, x86_64-linux-gnu-thread-multi
nvtype=double, nvsize=8, d_longdbl=define, longdblsize=16
libc=/lib/libc-2.3.6.so, libperl=libperl.so.5.8.8
0d22334455667708
Significand​: 6586440866800141.00000000
Significand​: 6586440866800142

* itanium.ens-lyon.fr
Perl 5.008008, linux 2.6.18-dsa-mckinley, ia64-linux-gnu-thread-multi
nvtype=double, nvsize=8, d_longdbl=define, longdblsize=16
libc=, libperl=libperl.so.5.8.8
0d22334455667708
Significand​: 6586440866800141.00000000
Significand​: 6586440866800142

* zaurus
Perl 5.006001, linux 2.2.19, arm-linux
nvtype=double, nvsize=8, d_longdbl=define, longdblsize=8
libc=/lib/libc-2.2.4.so, libperl=libperl.so.5.6.1
5566770811223344
Significand​: 6586440866800145.00000000
Significand​: 6586440866800145

* ay
Perl 5.008008, linux 2.6.8-powerpc-smp, powerpc-linux-gnu-thread-multi
nvtype=double, nvsize=8, d_longdbl=define, longdblsize=16
libc=/lib/libc-2.6.1.so, libperl=libperl.so.5.8.8
087766554433220d
Significand​: 6586440866800141.00000000
Significand​: 6586440866800141

* prunille
Perl 5.008008, darwin 8.8.0, darwin-2level
nvtype=double, nvsize=8, d_longdbl=define, longdblsize=16
libc=/usr/lib/libc.dylib, libperl=libperl.a
087766554433220d
Significand​: 6586440866800141.00000000
Significand​: 6586440866800141

* allo-psmn.ens-lyon.fr
Perl 5.00503, solaris 2.8, sun4-solaris
nvtype=, nvsize=, d_longdbl=define, longdblsize=16
libc=/lib/libc.so, libperl=libperl.so
0877665544332211
Significand​: 6586440866800145.00000000
Significand​: 6586440866800145

* bar.loria.fr
Perl 5.008, solaris 2.7, sun4-solaris
nvtype=double, nvsize=8, d_longdbl=define, longdblsize=16
libc=/lib/libc.so, libperl=libperl.so
087766554433220e
Significand​: 6586440866800142.00000000
Significand​: 6586440866800142

* td152.testdrive.hp.com
Perl 5.008008, freebsd 6.2-release, i386-freebsd-64int
nvtype=double, nvsize=8, d_longdbl=define, longdblsize=12
libc=, libperl=libperl.so
0d22334455667708
Significand​: 6586440866800141.00000000
Significand​: 6586440866800142

* td191.testdrive.hp.com
Perl 5.008008, hpux 11.00, PA-RISC1.1-thread-multi
nvtype=double, nvsize=8, d_longdbl=define, longdblsize=16
libc=/lib/libc.sl, libperl=libperl.a
087766554433220d
Significand​: 6586440866800141.00000000
Significand​: 6586440866800140

Perl Info

Flags:
    category=core
    severity=low

Site configuration information for perl v5.8.8:

Configured by Debian Project at Sun Nov  4 15:45:42 UTC 2007.

Summary of my perl5 (revision 5 version 8 subversion 8) configuration:
  Platform:
    osname=linux, osvers=2.6.22-3-amd64, archname=x86_64-linux-gnu-thread-multi
    uname='linux deneb 2.6.22-3-amd64 #1 smp thu oct 11 15:23:23 utc 2007 x86_64 gnulinux '
    config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN -Dcccdlflags=-fPIC -Darchname=x86_64-linux-gnu -Dprefix=/usr -Dprivlib=/usr/share/perl/5.8 -Darchlib=/usr/lib/perl/5.8 -Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5 -Dvendorarch=/usr/lib/perl5 -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl/5.8.8 -Dsitearch=/usr/local/lib/perl/5.8.8 -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3 -Dman1ext=1 -Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Uusesfio -Uusenm -Duseshrplib -Dlibperl=libperl.so.5.8.8 -Dd_dosuid -des'
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=define use5005threads=undef useithreads=define usemultiplicity=define
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=define use64bitall=define uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBIAN -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBIAN -fno-strict-aliasing -pipe -I/usr/local/include'
    ccversion='', gccversion='4.1.2 20061115 (prerelease) (Debian 4.1.1-21)', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -L/usr/local/lib'
    libpth=/usr/local/lib /lib /usr/lib
    libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt
    perllibs=-ldl -lm -lpthread -lc -lcrypt
    libc=/lib/libc-2.3.6.so, so=so, useshrplib=true, libperl=libperl.so.5.8.8
    gnulibc_version='2.3.6'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib'

Locally applied patches:
    


@INC for perl v5.8.8:
    /home/vlefevre/lib/site_perl/x86_64-linux-gnu-thread-multi
    /home/vlefevre/lib/site_perl
    /etc/perl
    /usr/local/lib/perl/5.8.8
    /usr/local/share/perl/5.8.8
    /usr/lib/perl5
    /usr/share/perl5
    /usr/lib/perl/5.8
    /usr/share/perl/5.8
    /usr/local/lib/site_perl
    .


Environment for perl v5.8.8:
    HOME=/home/vlefevre
    LANG=POSIX
    LANGUAGE (unset)
    LC_COLLATE=POSIX
    LC_CTYPE=en_US.UTF-8
    LC_TIME=en_DK
    LD_LIBRARY_PATH=/lib64:/home/vlefevre/x86_64/lib:/home/vlefevre/lib
    LOGDIR (unset)
    PATH=/opt/sge/bin/lx24-amd64:/home/vlefevre/bin:/home/vlefevre/x86_64/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/bin:/usr/games:.
    PERL5LIB=/home/vlefevre/lib/site_perl
    PERL_BADLANG (unset)
    SHELL=zsh

[p5pRT]

From @cpansprout

I think this is a duplicate of #41202, but I’m not sure.

[p5pRT]

@cpansprout - Status changed from 'new' to 'open'

[p5pRT]

From zefram@fysh.org

Father Chrysostomos wrote​:

I think this is a duplicate of #41202, but I'm not sure.

It's in two parts, one of which is [perl #41202], the other of which
is not. The two parts are best treated separately.

On the input side, the decimal literal is being converted incorrectly
to binary floating point. The closest available floating point
value is 0x1.7665544332211p-888, which in this test would show the hex
representation 1122334455667708, 5566770811223344, or 0877665544332211,
depending on endianness. The requestor is correct in identifying
which machines have converted this correctly. The other machines have
mostly produced 0x1.766554433220dp-888, with one 0x1.766554433220ep-888.
This part of the problem is just another instance of [perl #41202].

On output there's a separate problem. The scaled version of the
correct $x value, 0x1.7665544332211p+52, is 6586440866800145 in decimal.
The machines that converted correctly on input also converted to decimal
correctly for output. The scaled erroneous value 0x1.766554433220dp+52 is
6586440866800141 in decimal. All the machines that had this value output
it correctly with the %.8f format, but most of them then incorrectly
output 6586440866800142 with the %.0f format. A couple got it right,
and one output 6586440866800140. The one machine that had the erroneous
value 0x1.766554433220ep+52 output it correctly as 6586440866800142 with
both formats.

On my machine (amd64, Linux 3.16.0, glibc 2.19) I can reproduce
both problems with Perl 5.27.7. On input I get the erroneous value
0x1.766554433220dp-888, and on output I see the correct decimal value
for %.8f and the wrong decimal value for %.0f. Trying out the same
things in C, using libc directly, I see correct conversions both ways.
The problems are both specific to Perl.

Putting aside the input problem, since it's got its own ticket, the
output problem can be boiled down to this on current Perls​:

$ perl -lwe 'printf "%.1f\n%.0f\n", (0x1.766554433220dp+52) x 2'
6586440866800141.0
6586440866800142

This problem is not specific to the value resulting from the erroneous
input conversion in the original test; it also happens for any other value
with that exponent and the least-significant bit of the significand set,
including for the correct input value in the original test.

-zefram

[p5pRT]

From zefram@fysh.org

The output problem is a bug in the internal F0convert(), which is used to
optimise the special case of %.0f. The rounding algorithm in F0convert()
goes wrong in that exponent because it assumes that it can losslessly add
0.5 to any NV, which it can't in that exponent or higher. It only hits
that exponent because in higher exponents the unwanted floating-point
rounding always leaves the NV unchanged​: only in that exact exponent
can adding 0.5 have the effect of adding 1.

There's another bug in F0convert(), which is that it can't distinguish
signed zeroes and outputs them all as positive​:

$ perl -lwe 'printf "%-.0f %.0f\n", ($_) x 2 for 0.0, -0.0'
0 0
-0 0

I've fixed both of these bugs in commit
60fa466. This ticket can now be closed​:
one half is fixed, and the other half duplicates an older ticket.

-zefram

[khwilliamson]

Closed (finally) as Zefram suggested.