-
-
Notifications
You must be signed in to change notification settings - Fork 169
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Decimal numbers are not correctly retrieved #16
Comments
this issue is double sided issue
raised in issue#11 I expect type int64 and this script: but actually the server deal the same in both situation it send precision = 0 and scale = 0xFF I make a mistake when solve issue#11 by making scale=0 when scale=0xFF to solve the |
I was suspecting something like that. Using floats to represent integer leads to this. I have seen a problem with 123456.78 giving 123456.77999999998 with GO-ORA then ret = ret / powerTable[x][2] May bet the OCI library is doing differently for not messing the precision? |
I do a test to see where is the problem
if I run the sql code
the output for 61 and 62 is
but the output of the final result is:
this means that the problem is not in decodeDouble if I change datatype from float64 to float32
|
I find the problem:
I change it to
suppose scale more than 0x80 is impractical now the code work correctly either you use int64 or float64 |
It solve the 69 case, I have added Println in DecodeDouble. Inside it , right after line 176 ret was exactly 9.88. Witch is good I still think the problem is in multiple divisions by power of 10. |
yes you are correct may be this line is the source of problem can you convert to: and see if any problem happen with you |
I did it, but it has no impact. |
Try -10, got -11.02... |
please send me byte also to analyze like this: i am working on it and find the location of first problem
|
I have found an interesting article about how Oracle is storing numbers: select N, vsize(N), dump(N) from (
select 0 N from dual union
select 1 N from dual union
select 69 N from dual union
select 1008 N from dual union
select 123456.78 N from dual union
select -1 N from dual union
select -1008 N from dual union
select -123456.78 N from dual
) This gives:
Placed into a go program, the result is:
I'm also forking the project to add tests for this function, and try to help on resolution |
I am nearly find the problem and in the way to solve
here I update the length of input slice and suppose this update will be alive after function end but this is not happen now I am working on update all the code and rewrite DecodeDouble and test it |
I push the changes please test and inform me |
I did a naïve implementation of the function to limit the use of floats at the minimal. I'll create a push request with it and test code. // DecodeDouble decode Oracle binary representation of numbers into float64
//
// Some documentation:
// https://gotodba.com/2015/03/24/how-are-numbers-saved-in-oracle/
// https://www.orafaq.com/wiki/Number
func DecodeDouble(inputData []byte) float64 {
if len(inputData) == 0 {
return math.NaN()
}
if inputData[0] == 0x80 {
return 0
}
var (
negative bool
exponent int
mantissa int64
)
negative = inputData[0]&0x80 == 0
if negative {
exponent = int(inputData[0]^0x7f) - 64
} else {
exponent = int(inputData[0]&0x7f) - 64
}
buf := inputData[1:]
// When negative, strip the last byte if equal 0x66
if negative && inputData[len(inputData)-1] == 0x66 {
buf = inputData[1 : len(inputData)-1]
}
for _, digit100 := range buf {
digit100--
if negative {
digit100 = 100 - digit100
}
mantissa *= 10
mantissa += int64(digit100 / 10)
mantissa *= 10
mantissa += int64(digit100 % 10)
}
digits := 0
temp64 := mantissa
for temp64 > 0 {
digits++
temp64 /= 100
}
exponent = (exponent - digits) * 2
if negative {
mantissa = -mantissa
}
ret := float64(mantissa) * math.Pow10(exponent)
return ret
} |
please test my change first |
I read your message after having created the PR. Your implementation works too. |
ok I will get you code and comment my code |
Thanks! I suppose EncodeDouble should get your attention too. |
I need more test for all the driver to find all problem and fix |
Fix #16 Rewrite of DecodeDouble to limit floating point errors
still this code give problem |
I update the code and problem solved |
I have found that decimal numbers are retrieved as integer.
The following query
returns
0
when expecting something like0,6666666666666666666666666666666666666667
Here is the test code:
The text was updated successfully, but these errors were encountered: