Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SUMMARIZE returns error when query contains large values #9978

Closed
1 task done
jraymakers opened this issue Dec 13, 2023 · 4 comments · Fixed by #9985
Closed
1 task done

SUMMARIZE returns error when query contains large values #9978

jraymakers opened this issue Dec 13, 2023 · 4 comments · Fixed by #9985
Assignees

Comments

@jraymakers
Copy link
Contributor

What happens?

When summarize is used on a query with a result that contains values of some types close to the limits of their ranges, an error is returned.

To Reproduce

Running the following command in the CLI returns the error below:

summarize select 9223372036854775296;
Error: Invalid Input Error: Type DOUBLE with value 9.223372036854776e+18 can't be cast because the value is out of range for the destination type INT64

Equivalently, this produces the same error:

summarize select bigint from test_all_types();

Although note that the error occurs with the value 9223372036854775296, which is a bit smaller than the value 9223372036854775807 from test_all_types() (which also produces the error).

Note that the following (with the next smallest value from 9223372036854775296) does not return an error:

summarize select 9223372036854775295;

Also, negative values of some types return similar but different errors:

summarize select -9223372036854775808::bigint;
Error: Conversion Error: Type INT128 with value 9223372036854775808 can't be cast because the value is out of range for the destination type INT64

Note that the cast to BIGINT is needed to produce this error, and the following (with a value one step closer to zero) does not result in an error:

summarize select -9223372036854775807::bigint;

OS:

Mac OS X 13.5.2 (M2 chip)

DuckDB Version:

0.9.3-dev1527

DuckDB Client:

CLI

Full Name:

Jeff Raymakers

Affiliation:

MotherDuck

Have you tried this on the latest main branch?

I have tested with a main build

Have you tried the steps to reproduce? Do they include all relevant data and configuration? Does the issue you report still appear there?

  • Yes, I have
@hawkfish
Copy link
Contributor

This is triggered in ApproxQuantileScalarOperation::Finalize. Which is a bit ironic, because the name suggests that perfect accuracy is not the top priority here...

@hawkfish hawkfish self-assigned this Dec 13, 2023
@hawkfish
Copy link
Contributor

The last failure is not an error: The cast fails because the number is too large (which is what the error message says):

select -9223372036854775808::bigint;

hawkfish added a commit to hawkfish/duckdb that referenced this issue Dec 13, 2023
Since the result is approximate anyway, just clamp extreme values
instead of throwing.

fixes: duckdb#9978
fixes: duckdblabs/duckdb-internal#903
@jraymakers
Copy link
Contributor Author

jraymakers commented Dec 13, 2023

The last failure is not an error: The cast fails because the number is too large (which is what the error message says):

select -9223372036854775808::bigint;

Good point. There is still an error more like the original one with large negative numbers, though:

summarize select hugeint from test_all_types() limit 1;
Error: Invalid Input Error: Type DOUBLE with value -1.7014118346046923e+38 can't be cast because the value is out of range for the destination type INT128

@jraymakers
Copy link
Contributor Author

Sorry, didn't mean to close.

Mytherin added a commit that referenced this issue Dec 14, 2023
Issue #9978: Approximate Quantile Overflow
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants