When John Gustafson introduced unums, then posits (Type III) we thought the woes of floating point arithmetic were behind us.
Additional dynamic range, precision and the promise of accurate calculation with the help of a quire — what is not to like?
As it turns out, not everything is for the best in posit land.
Have a look at the illustration below. It is taken from John’s paper, Posit Arithmetic.
This wheel shows the values expressed by a rather small posit, which has just 5 bit overall length and 1 bit of exponent. Here you can observe a key feature of posits: tapered accuracy.
Numbers near 1 have more precision, while extremely big numbers and extremely small numbers have less.
However the illustration does not show clearly how much gap there really is between numbers at the extreme. Notice how numbers jump from 4 to 8, then 8 to 16, then 16 to 64 before going to infinity.
Larger posits can express more numbers, but what I call the race to infinity at the extreme of the range is the same. For example, the largest 32-bit posit with two bits of exponent is ≅1.33⨯1036. The second largest is ≅8.31⨯1034, a factor of 16 smaller. What kind of operation is there that produces such numbers, and in which cases does the difference between the two make an actual difference to the result?
It would be more useful to have more intermediate numbers even if not rising to such extreme exponents.
Posits do offer a mechanism to achieve the above, by reducing the exponent bits. A 32-bit posit with one bit of exponent has a largest value of ≅1.15⨯1018 while the next largest value is four times smaller, or ≅2.88⨯1017.
The next option is to reserve no bits for the exponent, resulting in a largest value of ≅1.07⨯109 with the next largest value twice as small. This is becoming a game of diminishing returns. There is no perfect value for es, the number of bits in the exponent.
At the other extreme, posits can express the exact reciprocal of these largest numbers. On the example in the picture they are 1/64 then 1/16. Zero is an exception value:
- If all bits are 0, the number represented is zero.
However, what to do if the result lands between 1/64 and 1/16? By reducing the exponent bits, the chasm will not be as wide but the dynamic range will also reduce. The smallest 32-bit number with one bit exponent is ≅8.67⨯10-19, as opposed to ≅7.52⨯10-37 with two bits for the exponent.
I will now introduce a number format with what I think is a better trade-off between dynamic range and the gap between extreme numbers.
I want to preserve most features of posits, such as no wasted bits, tapered accuracy and integer-logic comparison.
Let us start by observing that extreme posit values correspond to the bits of the regime taking the entire number length. In the illustration above, value 64 is composed of 4 bits regime, leaving just one sign bit in the 5-bit number. Similarly in the largest 32-bit number the regime occupies 31 bits.
Therefore my contribution is to introduce U, the maximum number of bits in the regime.
If the regime does not consume all bits at the extremity of the range there is an opportunity to introduce intermediate values by utilising the remaining bits.
The following relationship with nbits, the bit length of the number, applies:
U + es < nbits
As an exception, if all the bits of the regime and all the bits of the exponent are same (1 or 0), the gap between consecutive numbers is doubled. This replaces the exception for a number where all bits are 0 in standard posits. In that case the usual formula becomes:
(-1)s × useedk × ( 2e+1 × f - 2e+1-b ) where b is the numeral value of the regime and exponent bit.
For example, let us take nbits=5, es=1 and U=2. The numbers in the top-right quadrant of the illustration become:
+1000 → 1
+1001 → 1.5
+1010 → 2
+1011 → 3
+1100 → 4
+1101 → 6
+1110 → 8
+1111 → 16
The dynamic range has reduced, but the gap between extreme numbers has reduced too.
How does it translate to a 32-bit number?
Let us take nbits=32, es=2 and U=16. The largest number it can express is ≅2.77⨯1019. The gap to the next largest value is ≅2.25⨯1015. There is still four decimal digits precision at this extreme, in stark contrast with standard posits!
If the range is too narrow, increasing either es or U makes it wider. Increasing es has more effect on the dynamic range but reduces precision overall. Increasing U has less effect on the dynamic range and reduces precision at the extremes, to the point where it can become similar to standard posits. It has no effect on precision around number 1.
The largest 32-bit number with two bit exponent and the maximal regime size (U=29) is ≅1.04⨯1035. The gap to the next largest value is ≅2.08⨯1034. We are getting close to the level of standard posits where the gap is of the same order as the largest value.
If the exponent size is increased instead, with es=3 and U=16 the largest number is ≅5.10⨯1038 and the gap to the next number is ≅8.31⨯1034.
Posits extended with the U value are compatible with standard posits when es = 0 and U = nbits - 1.
A variation of this proposal allows the maximum bit size of the regime to take any value U < nbits, in which case standard posits are just the subset of extended posits where U = nbits - 1.