Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Result precision issue with Decimal data type #2969

Closed
monetdb-team opened this issue Nov 30, 2020 · 0 comments
Closed

Result precision issue with Decimal data type #2969

monetdb-team opened this issue Nov 30, 2020 · 0 comments

Comments

@monetdb-team
Copy link

@monetdb-team monetdb-team commented Nov 30, 2020

Date: 2012-01-10 13:00:39 +0100
From: sellam
To: SQL devs <>
Version: 11.7.5 (Dec2011) [obsolete]
CC: @drstmane

Last updated: 2012-02-29 10:34:27 +0100

Comment 16762

Date: 2012-01-10 13:00:39 +0100
From: sellam

User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.63 Safari/535.7
Build Identifier: 9e3ad41dfa21 (Aug2011)

Consider the following function:

create function bug (x decimal(7,5), y decimal(8,5)) returns decimal(15,0) begin
return power(10,7);
end;

We have the following:

select bug(12,12);
+------------------+
| bug_single_value |
+==================+
| 10000000 |
+------------------+

That's fine. However:

sql>select bug(12.1111,12.1111);
+-------------------+
| bug_single_value |
+===================+
| 1000.0000 |
+-------------------+

Reproducible: Always

hg id
9e3ad41dfa21 (Aug2011)

Comment 16763

Date: 2012-01-10 13:36:20 +0100
From: @sjoerdmullender

Changeset 475e900c55dc made by Sjoerd Mullender sjoerd@acm.org in the MonetDB repo, refers to this bug.

For complete details, see http//devmonetdborg/hg/MonetDB?cmd=changeset;node=475e900c55dc

Changeset description:

Added test for bug #2969.

Comment 16943

Date: 2012-02-19 18:21:00 +0100
From: @drstmane

Appears to be fixed in the default branch (only) after changesets
http://dev.monetdb.org/hg/MonetDB/rev/8ee024048e42
http://dev.monetdb.org/hg/MonetDB/rev/e2d3ecc7237c
http://dev.monetdb.org/hg/MonetDB/rev/03df27701cf7
http://dev.monetdb.org/hg/MonetDB/rev/70dc90e1de2f

Remains open/failing in the Dec2011 branch.

Comment 16946

Date: 2012-02-19 18:47:01 +0100
From: @drstmane

Fixed (also) in Dec2011 by transplanting changeset 70dc90e1de2f from default to Dec2011:
http://dev.monetdb.org/hg/MonetDB/rev/ec5692ecaf8c

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
1 participant