Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ROUND() does not allow second argument >=1 when its first argument is more than MAX_BIGINT / 10 [CORE6356] #6597

Closed
firebird-issue-importer opened this issue Jul 6, 2020 · 5 comments

Comments

@firebird-issue-importer
Copy link

@firebird-issue-importer firebird-issue-importer commented Jul 6, 2020

Submitted by: @pavel-zotov

Consider number: 922337203685477580 = floor( max_bigint / 10 )

This:

SQL> select round( 922337203685477580 ,1) from rdb$database;

-- works OK:

INPUT message field count: 0

OUTPUT message field count: 1
01: sqltype: 580 INT64 scale: 0 subtype: 0 len: 8
: name: ROUND alias: ROUND
: table: owner:

            ROUND

=====================
922337203685477580

The same number + 1 will fail:

SQL> select round( 922337203685477581,1) from rdb$database;

INPUT message field count: 0

OUTPUT message field count: 1
01: sqltype: 580 INT64 scale: 0 subtype: 0 len: 8
: name: ROUND alias: ROUND
: table: owner:

            ROUND

=====================
Statement failed, SQLSTATE = 22003
arithmetic exception, numeric overflow, or string truncation
-numeric value is out of range
SQL>

Commits: fd9600f

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented Jul 6, 2020

Modified by: @pavel-zotov

description: SQL> set sqlda_display on;

SQL> select round(170141183460469231731687303715884105727,1) from rdb$database;

INPUT message field count: 0

OUTPUT message field count: 1
01: sqltype: 32752 INT128 scale: 0 subtype: 0 len: 16
: name: ROUND alias: ROUND
: table: owner:

                                    ROUND

=============================================
Statement failed, SQLSTATE = 22003
arithmetic exception, numeric overflow, or string truncation
-numeric value is out of range
SQL>

But:

SQL> select round( cast(170141183460469231731687303715884105727 as int128),0) from rdb$database;

-- works fine:

INPUT message field count: 0

OUTPUT message field count: 1
01: sqltype: 32752 INT128 scale: 0 subtype: 0 len: 16
: name: ROUND alias: ROUND
: table: owner:

                                    ROUND

=============================================
170141183460469231731687303715884105727

=>

Consider number: 922337203685477580 = floor( max_bigint / 10 )

This:

SQL> select round( 922337203685477580 ,1) from rdb$database;

-- works OK:

INPUT message field count: 0

OUTPUT message field count: 1
01: sqltype: 580 INT64 scale: 0 subtype: 0 len: 8
: name: ROUND alias: ROUND
: table: owner:

            ROUND

=====================
922337203685477580

The same number + 1 will fail:

SQL> select round( 922337203685477581,1) from rdb$database;

INPUT message field count: 0

OUTPUT message field count: 1
01: sqltype: 580 INT64 scale: 0 subtype: 0 len: 8
: name: ROUND alias: ROUND
: table: owner:

            ROUND

=====================
Statement failed, SQLSTATE = 22003
arithmetic exception, numeric overflow, or string truncation
-numeric value is out of range
SQL>

summary: ROUND() of INT128 number requires is to be explicitly cased to INT128 => ROUND() does not allow second argument >=1 when its first argument is more than MAX_BIGINT / 10

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented Jul 7, 2020

Modified by: @AlexPeshkoff

assignee: Alexander Peshkov [ alexpeshkoff ]

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented Jul 8, 2020

Modified by: @AlexPeshkoff

status: Open [ 1 ] => Resolved [ 5 ]

resolution: Fixed [ 1 ]

Fix Version: 4.0 RC 1 [ 10930 ]

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented Jul 9, 2020

Modified by: @pavel-zotov

status: Resolved [ 5 ] => Resolved [ 5 ]

QA Status: No test => Done successfully

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented Jul 9, 2020

Modified by: @pavel-zotov

status: Resolved [ 5 ] => Closed [ 6 ]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants