hexfp may lose 1-3 low order bits (most often, 1)

[p5pRT]

Migrated from rt.perl.org#126586 (status was 'open')

Searchable as RT126586$

[p5pRT]

From @jhi

5.22.0, or bleadperl at 0c8adad​:

$ ./perl -wle 'print 0xf.ffffffffffffp0'
16
$ ./perl -wle 'print 0xf.ffffffffffff8p0'
Hexadecimal float​: mantissa overflow at -e line 1.
16
$ ./perl -wle 'print 0xf.ffffffffffffcp0'
Hexadecimal float​: mantissa overflow at -e line 1.
16

The thing is, the second one is not an overflow​: in the usual 53-bits-mantissa of IEEE doubles, that's exactly 53 bits. The third one has 54, so it correctly is an overflow. The first one is 52 bits, so not an overflow.

I noticed this case while staring for at the code for perl #126582. So the last hexdigit with bits straddling the NV_MANT_DIG (often 53, see above), may or may not lose some of its bits.

Note that this may be argued at least two ways​:

(1) if one specifies, say, 0x8, as the last hex digit, one may argue that one still specifies all the four bits, not just the top one.
(2) or one may argue that one only specified the top bit.

The code has so far been choosing the interpretation (1), but maybe the (2) makes more sense, because otherwise one may never specify all the 53 bits using the hexdigits, which is unfair the the very common case. (One could only specify up to 52 bits, using 13 hexdigits.) And hexfp is all about being able to specify the fp down to the last bit.

[p5pRT]

From @jhi

http​://perl5.git.perl.org/perl.git/commit/96524c28e76cdc1099eebc2832aef5b789ebbc11

Smoked all over the place, seems okay. (Testing more in double-double places would be nice.)

[p5pRT]

@jhi - Status changed from 'new' to 'resolved'

[p5pRT]

From @ap

* Jarkko Hietaniemi <perlbug-followup@​perl.org> [2015-11-07 23​:45]​:

Note that this may be argued at least two ways​:

(1) if one specifies, say, 0x8, as the last hex digit, one may argue
that one still specifies all the four bits, not just the top one.
(2) or one may argue that one only specified the top bit.

The code has so far been choosing the interpretation (1), but maybe
the (2) makes more sense, because otherwise one may never specify all
the 53 bits using the hexdigits, which is unfair the the very common
case. (One could only specify up to 52 bits, using 13 hexdigits.) And
hexfp is all about being able to specify the fp down to the last bit.

To the extent that my mathematical reasoning will carry me, there is no
practical loss of precision from losing trailing zero bits… *except* for
the most significant zero bit. Correct?

If so, then this is an irksome case, since only the most significant bit
from the last nibble can be retained in a 53-bit mantissa.

With the less significant bits all being 0, if the most significant bit
is also 0 then there is no loss of precision from dropping the trailing
bits.

However, if the most significant bit is 1 then losing the zero from the
*second*-most significant bit *is* an actual loss of precision! :-(

So you can specify a 13th hex digit with no loss of precision, but only
if that digit is 0. That’s… not terribly useful. It’s unfortunate that
this fact should make it impossible to specify the LSB in a 4x+1 bits
mantissa as 1.

These are options I can think of​:

1. Downgrade this case to a warning. Namely, merely warn about an extra
  nibble if mantissa length is not a multiple of 4, but all of the bits
  being lost are 0. Possibly even accept dropped 0 bits silently if the
  mantissa LSB is 0.

  I.e. a 53-bit mantissa 0000000000000 would be silent, 0000000000008
  would warn, and 000000000000C would error.

  I guess the warning would have to be in a special category of its own
  (`hexfpmantlsb`?) so that you could turn it off specifically. But it
  would be annoying to be forced to turn off the warning *every time*
  you want to specify a float down to the 53rd mantissa bit.

2. Add notation to specify the number of trailing zero bits that should
  be considered unspecified instead of explicitly zero.

  This would be kinda cryptic and doesn’t seem to have a real upside in
  explicitness. Intuitively I thought it should, but it doesn’t because
  the float literal doesn’t change length on different systems. A float
  literal with 13 hex digits is an error on a system with 52 or fewer
  bits of mantissa. And on a system with 56 or more bits of mantissa it
  can be represented exactly. So more precision than you intended can
  only be inadvertently lost when going from a system with e.g. 55 bits
  of mantissa to one with 53 bits​: both would require hex floats with
  the same number of digits to specify all their mantissa bits but they
  differ in how many bits from the last hex digit they can retain. Such
  minute differences in precision do not exist in practice.

Any others?

Regards,
--
Aristotle Pagaltzis // <http​://plasmasturm.org/>

[p5pRT]

From @jhi

Re-opening, since this case is not clear-cut, as pointed out.

So the commit I made makes the straddling case to warn, *if* they have one-bits beyond the 53th (if speaking about the common IEEE double case). But if there's just the right number of one-bits in the straddling nibble, no warning. (Nothing is erroring.)

But if one then continues to add more nibbles/hexdigits, of any content, the warning will again happen.

I think at least for 5.22.1 (*) the commit I made enhances the situation, since starting warning after the 13th nibble is just silly, the change effectively moved the warning one bit further.

FWIW, I think all this complication argues for officially (for 5.23/24) allowing the accidentally working "binfp", for allowing just the right number of bits. (The "octfp" is another matter, 3 bits presents even worse confusion than 4 bits.)

(*) Steve Hay told me that since these changes are fixing a new feature, they are worthy. (Especially the earlier perl #126582.)

[p5pRT]

@jhi - Status changed from 'resolved' to 'open'

[p5pRT]

From @jhi

With the less significant bits all being 0, if the most significant bit
is also 0 then there is no loss of precision from dropping the trailing
bits.

However, if the most significant bit is 1 then losing the zero from the
*second*-most significant bit *is* an actual loss of precision! :-(

So you can specify a 13th hex digit with no loss of precision, but only
if that digit is 0. That’s… not terribly useful. It’s unfortunate that
this fact should make it impossible to specify the LSB in a 4x+1 bits
mantissa as 1.

Regarding this​: isn't this about whether the conversion is truncating or rounding, and rounding which way? Now it's strictly truncating.

[p5pRT]

From @arc

Aristotle Pagaltzis <pagaltzis@​gmx.de> wrote​:

So you can specify a 13th hex digit with no loss of precision, but only
if that digit is 0. That’s… not terribly useful. It’s unfortunate that
this fact should make it impossible to specify the LSB in a 4x+1 bits
mantissa as 1.

These are options I can think of​:

1. Downgrade this case to a warning. Namely, merely warn about an extra
nibble if mantissa length is not a multiple of 4, but all of the bits
being lost are 0. Possibly even accept dropped 0 bits silently if the
mantissa LSB is 0.

2. Add notation to specify the number of trailing zero bits that should
be considered unspecified instead of explicitly zero.

Any others?

Does this work?

3. Treat all bits of all hex digits at the least significant end as
part of the user's intention, including trailing zero bits; but ignore
zero bits at the high end. Then specifying all 53 significand bits of
an IEEE-ish double could be done by always having a 1 as the leftmost
hex digit​:

0x1.123456789abcdP0 # no warning; rightmost bit == 1
0x11.23456789abcdP-4 # same
0x2.123456789abcdP0 # "mantissa overflow"

That is​: the first two examples specify exactly 53 bits after the
leftmost zeros, so they're fine; but the third tries to specify 54
bits after the leftmost zeros, so it overflows.

This seems consistent with the way we treat high zero bits in octal
integer literals​:

$ perl -wE '$_ = "01" . "0" x 20 . "7"; say for $_, sprintf "%#o", eval'
Octal number > 037777777777 non-portable at (eval 1) line 1.
01000000000000000000007
01000000000000000000007
$ perl -wE '$_ = "02" . "0" x 20 . "7"; say for $_, sprintf "%#o", eval'
Integer overflow in octal number at (eval 1) line 1.
Octal number > 037777777777 non-portable at (eval 1) line 1.
02000000000000000000007
01777777777777777777777

That is​: in the first example there, we have 3 bits in the trailing
"7", a further 20*3 == 60 bits in the middle zeros, and one bit after
the leftmost zeros in the leading "1", for a total of exactly 64 bits,
yielding no overflow on this 64-bit system. In the second example, we
have the same 63 bits starting at the zeros, and *two* bits after the
leftmost zero in the leading "2", for a total of 65 bits; so we get an
overflow.

(Hex integer literals seem to work the same way too, but they don't
have any leftover bits on 32- or 64-bit platforms, so there's no
partial-digit problem to deal with there.)

--
Aaron Crane ** http​://aaroncrane.co.uk/

[p5pRT]

From @ap

* Jarkko Hietaniemi via RT <perlbug-followup@​perl.org> [2015-11-08 15​:35]​:

Isn't this about whether the conversion is truncating or rounding, and
rounding which way?

It was only about me conflating things. But my confusion eludes me yet,
so please just disregard everything I said.

[jkeenan]

From @jhi

5.22.0, or bleadperl at 0c8adad​:

$ ./perl -wle 'print 0xf.ffffffffffffp0' 16 $ ./perl -wle 'print 0xf.ffffffffffff8p0' Hexadecimal float​: mantissa overflow at -e line 1. 16 $ ./perl -wle 'print 0xf.ffffffffffffcp0' Hexadecimal float​: mantissa overflow at -e line 1. 16

The thing is, the second one is not an overflow​: in the usual 53-bits-mantissa of IEEE doubles, that's exactly 53 bits. The third one has 54, so it correctly is an overflow. The first one is 52 bits, so not an overflow.

I noticed this case while staring for at the code for perl #126582. So the last hexdigit with bits straddling the NV_MANT_DIG (often 53, see above), may or may not lose some of its bits.

Note that this may be argued at least two ways​:
...

This ticket was still in heavy-discussion stage when it was unintentionally closed in the RT-to-GH transition in October 2019. Re-opening now. @jhi, @arc, @ap, please discuss. Thanks.