Support full range of `double` #128

Closed
stig opened this Issue Mar 21, 2012 · 16 comments

Comments

Projects
None yet
4 participants
Owner

stig commented Mar 21, 2012

(via @trevyn on issue #127)

Currently SBJson uses NSDecimalNumber for all floating-point numbers and integers of more than 19 digits. NSDecimalNumber has 38 digits of precision, but at the cost of supporting a smaller range than double. While double only supports ~17 digits of precision it supports exponents up to 308, compared to NSDecimalNumber's 127.

I'm currently torn between:

  1. Changing to use NSNumber (i.e. a double) for floating point numbers. This would support larger/smaller values than now, at the cost of lower precision.
  2. Try to intelligently use either double or NSDecimalNumber based on the length of the mantissa and/or exponent.

I suspect that 1) would be more in line with what people expect, particularly since JavaScript uses double as its underlying datatype.

stig pushed a commit that referenced this issue Mar 21, 2012

why are DECIMAL_EXPONENT_MIN and DECIMAL_EXPONENT_MAX set to very low absolute value?

Owner

stig commented Oct 14, 2012

Because the current implementation uses nsdecimalnumber and its implementation sacrificed range of max/min values for greatly increased precision. I'm currently leaning towards just using double, but I haven't had a chance to make the change yet.

This fix warrants a 3.2 release (or even 4.0) due to the precision changes it might cause.

Stig

Sent from my iPhone

On 14 Oct 2012, at 02:29, Thanapon Noraset notifications@github.com wrote:

why are DECIMAL_EXPONENT_MIN and DECIMAL_EXPONENT_MAX set to very low absolute value?


Reply to this email directly or view it on GitHub.

Thank you. You have a good point. I think I have to round a very small number to 0.0 for now.

Coeur commented Nov 5, 2012

If possible, best results for iOS might be:

  • NSDecimalNumber for 1-127 digits
  • NSNumber for 128-308 digits

We don't have to decrease the precision because of Javascript: Json is kind of a standard for REST programming used with .NET, PHP, Java, Ruby, etc.

stig added a commit that referenced this issue Jun 22, 2013

Move to less surprising handling of floating point numbers - fixes #113
…, #128 & #164

This removes the last trace of NSDecimalNumber, which has some surprising behaviour. It allows more precision but less range than double. This required:

* Removing tests for real numbers that cannot be represented accurately using IEEE floating point.
* Avoid rounding issues in tests when all we want is to make sure exponent is handled.

For clarity this also renamed tests for (positive and negative) zero for real numbers.

@stig stig closed this Jun 22, 2013

juhovh commented Aug 10, 2015

I really want my JSON parser to parse decimals as NSDecimalNumber instead of plain NSNumber, and I was hoping SBJson to be the solution but appears not. Is there any way to make this configurable?

juhovh commented Aug 10, 2015

And referring to the other issue, actually 17 digits of total precision would be enough for me, but I want absolute precision without roundings.

Owner

stig commented Aug 10, 2015

You could always patch SBJson locally to do that. If you know you don't need support for a range outside of what NSDecimalNumber supports it's a simple case of using it instead of the creating a double.

Another option may be to send the number as integer nominator/denominator parts of a fraction? As long as each integer is less than 21 digits SBJson will never round anything. You can then create a NSDecimalNumber yourself from those. That doesn't require any patching of SBJson.

juhovh commented Aug 10, 2015

Thanks for extremely quick response. Sending as nominator/denominator is something I have wondered myself, but then again that feels uglier than sending numbers as strings, which I wouldn't like to do either (but seriously need to consider). For now it would seem that the floating point precision is enough, but I think iOS would also need a JSON parser/serialiser that supports NSDecimalNumber as an option for cases that need it. I don't see that much point in using SBJson over native NSJsonSerialization class except for that...

Really too bad, since in Java (backend and Android) handling decimals can be easily made quite safe using BigDecimal class, and there are json-bignum implementation for JavaScript too for cases where precision is more important than performance. So iOS is my only problem.

Owner

stig commented Aug 10, 2015

I looked for a BigNum library for SBJson--and indeed I thought NSDecimalNumber was it for a while, until I realised how it actually worked!

juhovh commented Aug 10, 2015

Yeah, it's a bit lazy for Apple to create a NSDecimalNumber and not go all the way with a BigInteger implementation to accompany it, the small exponent range also comes from the quite limited integer part. But it's still closer to it than nothing, doubles are giving rounding errors quite often already for 2 digits, which is quite annoying.

Owner

stig commented Aug 10, 2015

If you want to provide a patch that can toggle between NSDecimalNumber and NSNumber (double) then I'm happy to review that -- and happy to work with you to get a PR tested. Open a new issue, perhaps?

juhovh commented Aug 10, 2015

I can open a new issue for a proper patch if I get to it, actually I have a bit too much on my list right now and mainly wondered was it removed for good or is there some way to get the old behaviour. And since SBJson4Parser has its different options (allowMultiRoot and unwrapRootArray) as init arguments instead of a single option bitmask which is the usual way to do this, it would be slightly difficult to add a toggle without breaking the existing API.

I'll just leave it here for now and try to get my app finished first, but writing it here for the record there are use cases where one would want to precisely parse for example -99.99, but do not need 10^x numbers where exponent goes to hundreds. :) The change without fallback was maybe a bit hasty.

Owner

stig commented Aug 10, 2015

I don’t think it was too hasty. The old implementation had very real problems — did you see #171 #171 ?

I would be open to add a constructor/initialiser with an option bit mask if that makes it easier. I did think about doing that, but didn’t bother since I only had two cases to support. They are both just wrappers around the designated initialiser though, so it shouldn’t be too hard to add. (Create new designated initialiser, delegate the current one to the new one with appropriate defaults: done.)

Stig

On 10 Aug 2015, at 20:22, juhovh notifications@github.com wrote:

I can open a new issue for a proper patch if I get to it, actually I have a bit too much on my list right now and mainly wondered was it removed for good or is there some way to get the old behaviour. And since SBJson4Parser has its different options (allowMultiRoot and unwrapRootArray) as init arguments instead of a single option bitmask which is the usual way to do this, it would be slightly difficult to add a toggle without breaking the existing API.

I'll just leave it here for now and try to get my app finished first, but writing it here for the record there are use cases where one would want to precisely parse for example -99.99, but do not need 10^x numbers where exponent goes to hundreds. :) The change without fallback was maybe a bit hasty.


Reply to this email directly or view it on GitHub #128 (comment).

Coeur commented Aug 11, 2015

Sorry, I will not be able to help at implementing my preference from 2.5 years ago (NSDecimalNumber for 1-127 digits, NSNumber for 128-308 digits), as it has been a long time I switched to native NSJsonSerialization. But good luck with that patch: your project is still a valuable one.

Coeur commented Aug 11, 2015

Oh, in Swift, we might be able work with Enum Associated Values:

enum MultiPrecisionAssociation {
case ShortNumber(NSDecimalNumber)
case LongNumber(Double)
}

juhovh commented Aug 11, 2015

I did see #171 and I agree it makes sense to process integers and decimals differently in the JSON parser, even though I fail to reproduce the off-by-one error in in Playground now. But that's not really an argument for or against the decimal side. But you're right that just adding an initialiser should be ok as well, I'll think about it.

@Coeur yeah the Swift enum would be quite elegant, although it still wouldn't work very well as default, because it causes quite much extra work to do a switch-case for each number to get the value. Not to mention It should include at least Int64 for integer values as well, which complicates it further.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment