plus() number type has more than 15 significant digits: 0.00005192067222222222

[hsiliev]

While trying to get the report for the current month I noticed this error is returned:
plus() number type has more than 15 significant digits: 0.00005192067222222222

The logs of the report app:

2015-11-20T13:31:16.20+0200 [RTR/0]      OUT abacus-usage-reporting.cfapps.staging.hanavlab.ondemand.com - [20/11/2015:11:31:15 +0000] "GET /v1/metering/organizations/610f6508-8b5d-4840-888d-0615ade33117/aggregated/usage/1446587999000 HTTP/1.1" 304 0 0 "https://abacus-report-app.domain/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:42.0) Gecko/20100101 Firefox/42.0" 192.168.0.107:58279 x_forwarded_for:"172.18.74.5" x_forwarded_proto:"https" vcap_request_id:a5d41e72-cd84-4452-61ad-669987408106 response_time:0.204214743 app_id:78bb85a2-9025-442b-bf2e-8ded90b3cfd3
2015-11-20T13:31:16.54+0200 [App/0]      OUT 2015-11-20T11:31:16.540Z e-abacus-router 610 Route error - generator error - { message: 'plus() number type has more than 15 significant digits: 0.00005192067222222222',
2015-11-20T13:31:16.54+0200 [App/0]      OUT   name: 'BigNumber Error' } - BigNumber Error: plus() number type has more than 15 significant digits: 0.00005192067222222222
2015-11-20T13:31:16.54+0200 [App/0]      OUT     at raise (/home/vcap/app/node_modules/bignumber.js/bignumber.js:1177:25)
2015-11-20T13:31:16.54+0200 [App/0]      OUT     at new BigNumber (/home/vcap/app/node_modules/bignumber.js/bignumber.js:261:50)
2015-11-20T13:31:16.54+0200 [App/0]      OUT     at BigNumber.P.plus.P.add (/home/vcap/app/node_modules/bignumber.js/bignumber.js:1790:17)
2015-11-20T13:31:16.54+0200 [App/0]      OUT     at sumCharges (/home/vcap/app/lib/index.js:87:38)
2015-11-20T13:31:16.54+0200 [App/0]      OUT     at iterator (/home/vcap/app/node_modules/underscore/underscore.js:184:16)
2015-11-20T13:31:16.54+0200 [App/0]      OUT     at /home/vcap/app/node_modules/underscore/underscore.js:199:14
2015-11-20T13:31:16.54+0200 [App/0]      OUT     at /home/vcap/app/lib/index.js:138:36
2015-11-20T13:31:16.54+0200 [App/0]      OUT     at _.map._.collect (/home/vcap/app/node_modules/underscore/underscore.js:172:24)
2015-11-20T13:31:16.54+0200 [App/0]      OUT     at /home/vcap/app/lib/index.js:137:34
2015-11-20T13:31:16.54+0200 [App/0]      OUT     at _.map._.collect (/home/vcap/app/node_modules/underscore/underscore.js:172:24)
2015-11-20T13:31:16.54+0200 [App/0]      OUT     at /home/vcap/app/lib/index.js:134:34
2015-11-20T13:31:16.54+0200 [App/0]      OUT     at _.map._.collect (/home/vcap/app/node_modules/underscore/underscore.js:172:24)
2015-11-20T13:31:16.54+0200 [App/0]      OUT     at chargeResource$ (/home/vcap/app/lib/index.js:112:29)
2015-11-20T13:31:16.54+0200 [App/0]      OUT     at tryCatch (/home/vcap/app/node_modules/babel-regenerator-runtime/runtime.js:61:40)
2015-11-20T13:31:16.54+0200 [App/0]      OUT     at GeneratorFunctionPrototype.invoke [as _invoke] (/home/vcap/app/node_modules/babel-regenerator-runtime/runtime.js:329:22)
2015-11-20T13:31:16.54+0200 [App/0]      OUT     at GeneratorFunctionPrototype.prototype.(anonymous function) [as next] (/home/vcap/app/node_modules/babel-regenerator-runtime/runtime.js:94:21)
2015-11-20T13:31:16.54+0200 [App/0]      OUT     at next (/home/vcap/app/node_modules/co/index.js:74:21)
2015-11-20T13:31:16.54+0200 [App/0]      OUT     at /home/vcap/app/node_modules/co/index.js:93:18
2015-11-20T13:31:16.54+0200 [App/0]      OUT     at Immediate._onImmediate (/home/vcap/app/node_modules/co/index.js:52:14)
2015-11-20T13:31:16.54+0200 [App/0]      OUT     at processImmediate [as _immediateCallback] (timers.js:367:17)
2015-11-20T13:31:16.54+0200 [RTR/0]      OUT abacus-usage-reporting.cfapps.staging.hanavlab.ondemand.com - [20/11/2015:11:31:15 +0000] "GET /v1/metering/organizations/610f6508-8b5d-4840-888d-0615ade33117/aggregated/usage HTTP/1.1" 500 0 90 "https://abacus-report-app.domain/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:42.0) Gecko/20100101 Firefox/42.0" 192.168.0.106:57547 x_forwarded_for:"172.18.74.5" x_forwarded_proto:"https" vcap_request_id:2190081d-e1d9-4638-4ec6-5b8d791233c8 response_time:0.558446134 app_id:78bb85a2-9025-442b-bf2e-8ded90b3cfd3
2

[cf-gitbot]

We have created an issue in Pivotal Tracker to manage this. You can view the current status of your issue at: https://www.pivotaltracker.com/story/show/108609786.

[hsiliev]

Seems like BigNumber does not allow by default more than 15 digits (as mentioned in the API doc):

Values of type number with more than 15 significant digits are considered invalid (if ERRORS is true) as calling toString or valueOf on such numbers may not result in the intended value.

Not sure, but I think the problem lies in this line:
https://github.com/cloudfoundry-incubator/cf-abacus/blob/master/lib/aggregation/reporting/src/index.js#L80

[hsiliev]

We might want to wrap the BigNumber and configure it to have only 15 significant digits.

If we want to keep the benefits of BigNumber we might want to keep all the digits in the DB and return only 15 to clients. Another approach would be to force them to use BigNumber?

Another approach would be to check if the number is BigNumber or default JS type number and based on that construct a new BNumber or just reuse the existing one.

WDYT?

[hsiliev]

It seems we are hitting a restriction of BigNumber - it supports only 15 significant digits to keep precision. However there are numbers that have 16 or more digits in JS. Plain ints for example have 16:

var biggestInt = 9007199254740992;
var smallestInt = -9007199254740992;

This github issue mentions that this is done on purpose and suggests two solutions:

1. turn off error mesages
2. convert numbers to string first

[hsiliev]

Disabling errors allowed me to get the report from Abacus.

[jsdelfino]

If I understand the BigNumber doc correctly, it is correct that Javascript supports 16 significant digits but some of these numbers will cause precision errors in floating point calculations and that's the reason why BigNumber introduced that limitation to 15 significant digits.

Like you said the recommendation from BigNumber is to either turn off error messages or convert numbers to strings first. I'd rather just turn off the error message as that's less disruptive and passing strings doesn't make BigNumber support more than 15 digits either anyway. So I'd suggest to call BigNumber.config({ ERRORS: false }) in the only two places where we require it (the resource config and reporting modules).

I'm not sure that we need to do that BigNumber.config in the tests as the numbers these tests will be seeing shouldn't have more than 15 significant digits (as they'll have been produced by computations over BigNumbers in the first place).

Makes sense?