Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal: Go 2: Add Posit data type #33205

Open
jfcg opened this issue Jul 20, 2019 · 17 comments
Open

Proposal: Go 2: Add Posit data type #33205

jfcg opened this issue Jul 20, 2019 · 17 comments

Comments

@jfcg
Copy link

@jfcg jfcg commented Jul 20, 2019

Posits are gaining traction as a replacement and clean improvement over IEEE 754. They offer reasonable dynamic ranges with better accuracy / energy efficiency / speed on many applications. They will probably replace IEEE floats in 5 to 10 years in cpus / gpus / tpus. They have an official standard.

Adding (experimental) support for posits will position Go 2 in a better place for developers' consideration who want to get better results out of their mathematical calculations in AI, ML, HPC, weather forecasting, astrophysics etc. areas in addition to typical backend programming that Go is already good at. This will possibly enable Go's penetration into scientific computing.

At the moment there is no hw architecture that supports posits, but there are many sw and fpga implementations. Initially it could start like soft-floats. As hw support becomes available, people will reap the benefits. Most floating-point code is readily convertable to posits: just rename data types.

Adding posits is non-trivial and a BIG step. It has to be done incrementally. Also there will possibly be a transition period where hw platforms will have IEEE floats and / or posits.

These are the (incomplete) steps of this proposal:

  • add posit8, posit16, posit32, posit64 data types as per the standard
  • add conversions from/to other data types
  • add "printing/scanning posits" functionality to fmt library
  • add math/posit standard library with standard Sine/Tan etc. (posit32)
  • add cxposit32, cxposit64, cxposit128 complex number data types
  • add math/cmplx/cxposit standard library (cxposit64)

There is a lot to do, so when hw support arrives, Go 2 will be ready to deliver ;)

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Jul 21, 2019

This is interesting but it's not obvious to me that this is an area where Go needs to lead. It would be painful if we added posits to the language and they didn't catch on.

I think the right first step is a third party package that implements them in a Go callable form, even though the normal operators would have to be implemented as methods.

@smasher164
Copy link
Member

@smasher164 smasher164 commented Jul 21, 2019

Even apart from the risk of adding it to the language, as it stands, I'm not sure that the posit standard is a drop-in replacement for floating-point as the paper claims. The fact that there are no signed infinities will break calculations that depend on that property.

More importantly, the fact that the standard doesn't specify a NaN value means that errors like 0.0 / 0.0 have to be signaled separately. Errors cannot quietly propagate through calculations like with qNaNs, so in Go that means either panicking or returning an error.

I would definitely want to try a library implementation like Ian mentioned, since there is no rush to preemptively support a datatype even before the hardware catches up.

@jfcg
Copy link
Author

@jfcg jfcg commented Jul 21, 2019

Posit does have a NaR (not a real number) value that represents IEEE NaN, 0/0, ±inf, etc.
There is a comprehensive analysis here, also comparing the two.

@matt2909
Copy link
Contributor

@matt2909 matt2909 commented Jul 21, 2019

Another piece, for balance (not by the author of posit, but by a leading numerics group).

https://hal.inria.fr/hal-01959581v3/document

from the conclusion:

"When posits are better than floats of the same size, they provide one or two extra digits of accuracy [19]. When they are worse than floats, the degradation of accuracy can be arbitrarily large. This simple observation should prevent us from rushing to replace all the floats with posits."

@jfcg
Copy link
Author

@jfcg jfcg commented Jul 21, 2019

True, that is a fundamental feature of posits. They provide better accuracy for typical numbers, and worse accuracy for very large/small magnitude numbers.

@smasher164
Copy link
Member

@smasher164 smasher164 commented Jul 21, 2019

Posit does have a NaR (not a real number) value that represents IEEE NaN, 0/0, ±inf, etc.

Interesting. It seems Gustafson justifies the NaR by arguing that

As with nonstandard rounding modes, a need to distinguish underflow, overflow, and NaN is generally an indication that an application is still in development and not yet ready for production use.

I'm not sure how often this principle is respected in practice, but this and the lack of subnormals greatly simplifies arithmetic. However, this proposal should either be put on hold or declined, since we should first experiment outside of library before moving forward. Further discussion should probably be on a mailing list.

@bcmills
Copy link
Member

@bcmills bcmills commented Jul 22, 2019

Why posits rather than, say, intervals or the more interval-like valids?

@jfcg
Copy link
Author

@jfcg jfcg commented Jul 23, 2019

From IEEE floats to interval arithmetic is probably too much to swallow for general community, so posits are the intended replacement by its designers, I think.

@bcmills
Copy link
Member

@bcmills bcmills commented Jul 30, 2019

Hrm. I'm not sure why they would expect two breaking ABI (and semantic) changes to be more acceptable than one change to the more desirable end state.

@jfcg
Copy link
Author

@jfcg jfcg commented Jul 31, 2019

I didn't get what you mean, was it for another issue?

@bcmills
Copy link
Member

@bcmills bcmills commented Jul 31, 2019

No, I mean that I don't see the point in adding posits and getting folks to migrate if there is something better — perhaps interval-based — just over the horizon.

@jfcg
Copy link
Author

@jfcg jfcg commented Aug 25, 2019

Here authors describe the unum evolution, posits are the 3rd generation and their proposed replacement for IEEE 754.

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Aug 27, 2019

There is no reason for Go to be an early adopter. Let's wait until there is a clear move toward posits before adding it to the language. Putting the issue on hold for now. In the meantime, a third party package would be an interesting idea.

@soypat
Copy link

@soypat soypat commented Jun 14, 2021

http://www.johngustafson.net/pdfs/BeatingFloatingPoint.pdf

Pretty comprehensive read on the ins-and-outs of posits along with comparisons to IEEE-754.

@JAicewizard
Copy link

@JAicewizard JAicewizard commented Jun 15, 2021

I have a semi-performant posit library over at https://gitlab.com/amfiremage/gosit. This, of course, performs nowhere as close as hardware floating points from the standard library, but most operations are reasonably fast compared to rusts implementation of soft-posit. Only the basic bath operations (*,/,+,-,sqrt), floor, and round (but no ceil) are implemented. No 64 bit posits are supported, since some operations would require 128 bit ints.

Since people asked for a third-party library first, I thought I would link mine. It is fuzzed against softposit-rs, and even found some bugs in their implementation. As long as the application can do with these basic operators, anyone can test this out in their application. Just dont expect a performance improvement. Maybe when you are heavily band-width limited and you can use 32 bits instead of 64, but that's probably rare.

@JAicewizard
Copy link

@JAicewizard JAicewizard commented Jun 15, 2021

No, I mean that I don't see the point in adding posits and getting folks to migrate if there is something better — perhaps interval-based — just over the horizon.

Interval algorithmic is not fundamentally better, it is more expensive and takes more time. It can also often give you one single answer, so things like ordering, cant really be defined. And besides, if you have interval algorithmic, you need a format for the min/max. With valids this would be posits, you need posits to have valids. Not exposing them to the users is something I would be against.

@soypat
Copy link

@soypat soypat commented Jun 15, 2021

From the framing of this issue it sounds to me that the question is not whether this change would be 'breaking' as some suggest. The issue suggests adding new types and not replacing floats (my understanding). I'd imagine any package with posit64 as an identifier would benefit greatly from a built-in. 🙂

Now that we have 'gosits' it is a good time to start playing around with them and test how they fare in action and keep a lookout for posit (or intervals, if that be the case) movements in the hw world.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
8 participants