float.hex discards sign from -nan

[sree314]

BPO	42210
Nosy	@tim-one, @rhettinger, @mdickinson, @vedgar

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-10-31.12:25:13.629>
created_at = <Date 2020-10-30.16:04:29.810>
labels = ['interpreter-core', 'type-bug', 'invalid']
title = 'float.hex discards sign from -nan'
updated_at = <Date 2020-11-02.17:14:37.527>
user = 'https://bugs.python.org/sree314'

bugs.python.org fields:

activity = <Date 2020-11-02.17:14:37.527>
actor = 'sree314'
assignee = 'none'
closed = True
closed_date = <Date 2020-10-31.12:25:13.629>
closer = 'mark.dickinson'
components = ['Interpreter Core']
creation = <Date 2020-10-30.16:04:29.810>
creator = 'sree314'
dependencies = []
files = []
hgrepos = []
issue_num = 42210
keywords = []
message_count = 5.0
messages = ['379979', '380007', '380046', '380055', '380239']
nosy_count = 5.0
nosy_names = ['tim.peters', 'rhettinger', 'mark.dickinson', 'veky', 'sree314']
pr_nums = []
priority = 'normal'
resolution = 'not a bug'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue42210'
versions = ['Python 3.6']

[sree314]

float.hex(float.fromhex("-nan")) returns 'nan'.

PyOS_double_to_string seems to check only if it is a NaN, but ignores the sign (unlike its handling of inf a few lines below):

<https://github.com/python/cpython/blob/e9e7d284c434768333fdfb53a3663eae74cb995a/Python/pystrtod.c#L903\>

bpo-5981 (https://bugs.python.org/issue5981) added the ability for fromhex to recognize -nan, but hex should probably have been updated at the same time.

Encountered when testing float implementations.

[rhettinger]

Having fromhex() support signed NaNs was reasonable because it is a valid possible input; however, the case isn't as clear-cut for output. Since applications are allowed to ignore the sign bit, no application should rely on seeing a particular sign for a NaN.

Also, existing applications may already have NaNs where the sign bit is set. If we change the output, we could break their published examples and doctests. So, this proposal may cause more harm than good.

[vedgar]

inf and -inf are really two different values (in the scope of the standard). Same as 5. and -5., or even 0. and -0. They behave differently in some exactly specified operations, and it is useful.

Are there any exactly specified operations whose specifications require the unequal treatment of nans with + and - signs? I don't think so, since it goes against the whole idea of nan as an unspecified number. (If you want to track its _history_, that's what payload is for.) My interpretation is that nan can have a sign bit only so unary minus and absolute value can be implemented quicker (without having to check whether the input is nan) -- not because it carries any useful semantics within the standard.

[mdickinson]

This isn't a bug: it was a deliberate decision, just like the choice to represent -nan as nan in float.__repr__ was a deliberate decision. NaNs don't have a meaningful sign - they have a sign bit, but it's best to regard that as just an extra bit of metadata (like the payload bits).

IEEE 754 explicitly refuses to interpret the sign bit of a NaN: section 1.4 of the 2019 version of the standard says: "This standard does not specify [...] Interpretation of the sign and significand fields of NaNs."

As Vedran points out, infinities are a very different beast: the difference between negative infinity and positive infinity matters.

[Raymond]

no application should rely on seeing a particular sign for a NaN

Yep, exactly.

[sree314]

Thanks, all. I just wanted to know if this was a deliberate decision or an oversight. It also took me a while to realize it washex, and not fromhex.

That allows the current behaviour to be easily worked around in Python code, and a backwards compatible optional parameter to hex() might allow -nan in the future without breaking existing software.

Surprisingly, the standard does allow this [but this may not have been the intent] -- (Section 6.2) "Recognize that format conversions,
including conversions between supported formats and external representations as character sequences,
might be unable to deliver the same NaN." The conversion to character sequences text is key since I'm using a test harness written in Python that writes out and reads back text files containing float data.

Section 6.3 in the standard also adds more details on when the sign bit in NaNs is relevant.