Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhancement in precision of calculations with NUMERIC/DECIMAL [CORE4409] #4731

Closed
firebird-issue-importer opened this issue Apr 28, 2014 · 16 comments

Comments

@firebird-issue-importer

Submitted by: @WarmBooter

Jira_subtask_outward CORE5525

Attachments:
decfloat.gbak
decFuncTest.sql

Votes: 2

The way Firebird deals with the precision of the results of calculations involving multiple arguments can easily cause overflows, specially in maths/divs, when the precision of the result is the sum of the precision of the arguments, what can easily extrapolate the actual limits of the currently numeric/decimal datatypes. Also, some care is needed in the intermediate calculations, to avoid loss of precision and overflows.

====== Test Details ======

::: NB :::
After fix CORE5700 ("DECFLOAT underflow should yield zero instead of an error"), 02-feb-2018, expected result was
changed: all expressions with "almost zero" result should NOT raise any error.

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented Dec 11, 2014

Modified by: @dyemanov

assignee: Dmitry Yemanov [ dimitr ]

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented Mar 9, 2016

Modified by: @dyemanov

Fix Version: 4.0 Alpha 1 [ 10731 ]

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented Sep 1, 2016

Modified by: @dyemanov

Fix Version: 4.0 Beta 1 [ 10750 ]

Fix Version: 4.0 Alpha 1 [ 10731 ] =>

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented Feb 27, 2017

Commented by: @AlexPeshkoff

Samples of decimal float data

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented Feb 27, 2017

Modified by: @AlexPeshkoff

Attachment: decfloat.gbak [ 13073 ]

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented Mar 8, 2017

Commented by: @pavel-zotov

How to restore database 'decfloat.gbak' ?

C:\MIX\firebird\fb40\gbak -z -c -v C:\MIX\firebird\QA\fbt-repo\tmp\c4409-decfloat.gbak /:C:\MIX\firebird\QA\fbt-repo\tmp\c4409-decfloat.fdb 1>c4409-decfloat.log 2>&1

Log:

gbak:gbak version WI-T4.0.0.550 Firebird 4.0 Unstable
gbak:opened file C:\MIX\firebird\QA\fbt-repo\tmp\c4409-decfloat.gbak
gbak:transportable backup -- data in XDR format
gbak: backup file is compressed
gbak:backup version is 11
gbak: Version(s) for database "/:C:\MIX\firebird\QA\fbt-repo\tmp\c4409-decfloat.fdb"
Firebird/Windows/Intel/i386 (access method), version "WI-T4.0.0.550 Firebird 4.0 Unstable"
Firebird/Windows/Intel/i386 (remote server), version "WI-T4.0.0.550 Firebird 4.0 Unstable/tcp (csprog)/P15"
Firebird/Windows/Intel/i386 (remote interface), version "WI-T4.0.0.550 Firebird 4.0 Unstable/tcp (csprog)/P15"
on disk structure version 13.0
gbak:created database /:C:\MIX\firebird\QA\fbt-repo\tmp\c4409-decfloat.fdb, page_size 8192 bytes
gbak:started transaction
gbak:restoring domain RDB$1
gbak:restoring domain RDB$2
gbak:restoring domain RDB$3
gbak:restoring table DTST
gbak: restoring column X
gbak:restoring table QTST
gbak: restoring column X
gbak: restoring column C
gbak:committing metadata
gbak: ERROR:unsuccessful metadata update
gbak: ERROR: DTST
gbak:Exiting before completion due to errors

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented Mar 16, 2017

Commented by: @AlexPeshkoff

To restore that database one should use decfloat branch.

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented Mar 16, 2017

Modified by: @dyemanov

assignee: Dmitry Yemanov [ dimitr ] => Alexander Peshkov [ alexpeshkoff ]

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented Mar 22, 2017

Modified by: @AlexPeshkoff

Attachment: decfloat.gbak [ 13073 ] =>

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented Mar 22, 2017

Commented by: @AlexPeshkoff

1. Updated sample. Gbak format now matches current state of decfloat branch, this format is expected to become permanent one.
2. Script with samples of various decfloat specific functions is added.

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented Mar 22, 2017

Modified by: @AlexPeshkoff

Attachment: decfloat.gbak [ 13079 ]

Attachment: decFuncTest.sql [ 13080 ]

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented Oct 16, 2017

Commented by: @AlexPeshkoff

Resolved with PR #⁠108 #108

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented Oct 16, 2017

Modified by: @AlexPeshkoff

status: Open [ 1 ] => Resolved [ 5 ]

resolution: Fixed [ 1 ]

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented Oct 17, 2017

Modified by: @pavel-zotov

status: Resolved [ 5 ] => Resolved [ 5 ]

QA Status: Done successfully

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented Oct 17, 2017

Modified by: @pavel-zotov

status: Resolved [ 5 ] => Closed [ 6 ]

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented Aug 12, 2018

Modified by: @pavel-zotov

status: Closed [ 6 ] => Closed [ 6 ]

Test Details: ::: NB :::
After fix CORE5700 ("DECFLOAT underflow should yield zero instead of an error"), 02-feb-2018, expected result was
changed: all expressions with "almost zero" result should NOT raise any error.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment