Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversion from numeric literals to DECFLOAT adds precision that is not originally present [CORE5697] #5963

Closed
firebird-issue-importer opened this issue Jan 6, 2018 · 8 comments

Comments

@firebird-issue-importer
Copy link

@firebird-issue-importer firebird-issue-importer commented Jan 6, 2018

Submitted by: @mrotteveel

Depends on CORE6060

When inserting into a table with a DECFLOAT(16) column,

Using

insert into decfloat16 (column1) values (1E300);
insert into decfloat16 (column1) values ('1E300');

and then select cast(column1 as varchar(30)) from decfloat16 will result in:

1.000000000000000E+300
1E+300

In other words, the conversion from literal adds 15 digits of precision that weren't actually present in the literal. The value resulting from string conversion is what I expect to get from the literal.

Commits: 5d70d4a

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented Jan 6, 2018

Modified by: @mrotteveel

description: When inserting into a table with a DECFLOAT(16) column,

Using

insert into decfloat16 (column1) values (1E300);
insert into decfloat16 (column1) values ('1E300');

and then select cast(column1 as varchar(30)) from decfloat16 will result in:

1.000000000000000E+300
1E+300

In other words, the conversion from literal adds 15 digits of precision that weren't actually present in the literal.

=>

When inserting into a table with a DECFLOAT(16) column,

Using

insert into decfloat16 (column1) values (1E300);
insert into decfloat16 (column1) values ('1E300');

and then select cast(column1 as varchar(30)) from decfloat16 will result in:

1.000000000000000E+300
1E+300

In other words, the conversion from literal adds 15 digits of precision that weren't actually present in the literal. The value resulting from string conversion is what I expect to get from the literal.

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented Jan 6, 2018

Modified by: @mrotteveel

Version: 4.0 Alpha 1 [ 10731 ]

Component: Engine [ 10000 ]

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented Jan 6, 2018

Modified by: @AlexPeshkoff

assignee: Alexander Peshkov [ alexpeshkoff ]

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented May 6, 2019

Modified by: @AlexPeshkoff

Link: This issue depends on CORE6060 [ CORE6060 ]

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented Jun 22, 2019

Commented by: @mrotteveel

This works OK now.

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented Jul 4, 2019

Modified by: @AlexPeshkoff

status: Open [ 1 ] => Resolved [ 5 ]

resolution: Fixed [ 1 ]

Fix Version: 4.0 Beta 2 [ 10888 ]

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented Jul 6, 2019

Modified by: @pavel-zotov

status: Resolved [ 5 ] => Resolved [ 5 ]

QA Status: No test => Done successfully

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented Jul 6, 2019

Modified by: @pavel-zotov

status: Resolved [ 5 ] => Closed [ 6 ]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants