Add support for hexadecimal float literals

[boggle]

We need to be able to parse the output of printf's %a for proper support of mathematical constants without loss of precision
(i.e. 0x1.fffffffffffffp+1023_f64 syntax)

[lifthrasiir]

I have played with this issue and concluded that this unfortunately revives issue #1306. The current lexer prohibits any alpha character after . because of it. We may choose selectively allow them when the pattern 0x<digits>. is found, but it seems too inconsistent and may require infinite lookahead if we want to elide an inconsistency.

[graydon]

I suggest requiring a digit or 0x or 0b after the dot. This avoids the collision and (curiously) lets you switch radix mid-literal if you choose. And it is only slightly uglier than what C99 makes you write.

[lifthrasiir]

@graydon Or we can require only zeroes after the dot. (e.g. 0x1fffffffffffff.0p+972_f64 instead of 0x1.fffffffffffffp+1023_f64)

[pcwalton]

I don't believe this is backwards incompatible, renominating.

[graydon]

accepted for feature-complete milestone

[pnkfelix]

visited for triage dating from 2013-07-15. Have we actually settled on a syntax here? This seems like it might be a nice easy task for a new contributor if we can hand them a clear specification for the syntax; but if the syntax is still in the design phase, then that claim is less true.

[graydon]

I think we haven't quite nailed the syntax but I should point out it'd be necessary to avoid colliding with suffixes (some of which start with f, a hex digit), so I think if we do this it should only work for specifying the mantissa, and only when combined with a full exponent (in decimal).

[thestinger]

This has yet to be implemented.

[catamorphism]

Not 1.0

[anatol]

Hexadecimal floats are quite popular among C math libraries. And I would love to use them in Rust as well.

I see that hexadecimal float has been implemented in rust as an extension, then moved to a separate crate and now it is at the rust-deprecated and fails to compile with nightly rust.

What is the future of this feature?

[ColonelThirtyTwo]

Also interested in this. I'm looking to implement a replayable physics system for a game, which requires consistent results across computers. Hex float literals would allow me to write bit-exact floating-point values in tests and constants.

At the very least, a statement why this feature is being deprecated would be appreciated, as this thread is the first result I get for "rust hex float literal".

[lfairy]

The situation is a bit icky at the moment. Procedural macros are going through a revamp right now (#38356), and so it would at least be a waste of time to keep hexfloat up-to-date while this is happening. But I don't know what the story will be after that.

[lifthrasiir]

If my understanding is correct, hexadecimal floats in C/C++ are there because decimal floats are not guaranteed to round correctly [1]. In Rust, however, after #27307---I suspect this is not necessarily intentional though!---almost all decimal floats (#31407 describes edge cases that are practically irrelevant) should round to the nearest, so you can just give rustc an appropriate number (say, 30) of fractional digits and will get the correctly rounded number. This would be a "practical" answer for now.

One thing for which I still think hexadecimal floats are relevant is a conversion from C/C++. You wouldn't want to convert all hexadecimal floats to decimal yourself :) I have recently written an experimental procedural macro that does exactly this, converting hexadecimal floats to decimal floats (that rustc can understand), but had been reluctant to release one because the status quo was not actually guaranteed so far---it's a pure luck in my opinion. If there is a way to construct a float with exact bit pattern only from constexprs I will adapt that.

[1] For example, ISO C99 only requires that decimal floats should be converted to a representable number within ±1.5 ulps ("the result is either the nearest representable value, or the larger or smaller representable value immediately adjacent to the nearest representable value").

[jameysharp]

@lifthrasiir, sooner or later I'd like to fix jameysharp/corrode#73 by correctly translating C hex floats to Rust, so I'd like to understand your comment better. I haven't done enough reading on floating-point issues to understand what's involved here yet.

Are you saying that, modulo Rust compiler bugs, every hex-float literal can be converted to a decimal float literal that the Rust compiler will convert to the same bit-pattern? If so, I'd be happy to have Corrode do that conversion. Can you recommend a reference I should read for an algorithm to do that conversion correctly? (A pointer to your procedural macro would be great, but ideally I'd like to have a paper or book to cite too.)

That said, I gather the exact decimal version of a hex float may take significantly more digits (right?), so perhaps the hex-float version is easier to read and understand, at least for people who care enough about numeric precision to use them. If hex floats are the human-preferred form for these numbers, I'd argue that Rust should support them. So IMO, more feedback from people who have used hex floats would help here.

[lifthrasiir]

Are you saying that, modulo Rust compiler bugs, every hex-float literal can be converted to a decimal float literal that the Rust compiler will convert to the same bit-pattern?

My belief is yes.

Can you recommend a reference I should read for an algorithm to do that conversion correctly?

You don't have to implement that, because the current Rust compiler and standard library implements all necessary algorithms (which is harder one :-). If you really need references see the following:

• The conversion from binary to decimal ("flt2dec", #24612) is a hybrid of [Dragon4] and [Grisu3].

  • [Dragon4]: Burger, R. G. and Dybvig, R. K. 1996. Printing floating-point numbers quickly and accurately. SIGPLAN Not. 31, 5 (May. 1996), 108-116.
  • [Grisu3]: Florian Loitsch. 2010. Printing floating-point numbers quickly and accurately with integers. SIGPLAN Not. 45, 6 (June 2010), 233-243.

• The conversion from decimal to binary ("dec2flt", #27307) is a couple of algorithms described by [Clinger]. (I haven't implemented them myself so my knowledge for them is limited, however.)

  • [Clinger]: William D. Clinger. 1990. How to read floating point numbers accurately. SIGPLAN Not. 25, 6 (June 1990), 92-101.

For the record, my implementation is lifthrasiir/hexf (published right now, not yet in crates.io). Feel free to pick up.

That said, I gather the exact decimal version of a hex float may take significantly more digits (right?), [...] If hex floats are the human-preferred form for these numbers, I'd argue that Rust should support them. [...]

Not exactly, for example, 0x1.999999999999bp-4 = 0.10000000000000002. I have no opinion on whether Rust should support hex floats or not.

[lifthrasiir]

I've now formally published hexf to crates.io. @jameysharp, I think you can probably use hexf-parse for your work? (The syntax without underscores should be identical almost same to C99 hexadecimal-floating-constant non-terminal sans optional floating-suffix.)

Edit: Aargh I missed one case. 0x1p1 should be valid but hexf doesn't recognize it; probably it is easy to account for, though.