Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

overflow error when counting very large tables in sql server #1498

Open
edward-burn opened this issue May 14, 2024 · 0 comments
Open

overflow error when counting very large tables in sql server #1498

edward-burn opened this issue May 14, 2024 · 0 comments

Comments

@edward-burn
Copy link
Contributor

In sql server with a very large table with 6 billion rows the below error occurs because COUNT is being used rather than COUNT_BIG. I was able to fix this by changing the translation to COUNT_BIG main...oxford-pharmacoepi:dbplyr:sql_server_count_big, but I'm not sure @hadley @mgirlich what you would think about switching to using this in dbplyr for sql server? The nice thing with using COUNT_BIG is that it should always work, but it does return a bigint data type rather than int as now https://learn.microsoft.com/en-us/sql/t-sql/functions/count-big-transact-sql?view=sql-server-ver16

Error in dplyr::collect():
! Failed to collect lazy table.
Caused by error:
! nanodbc/nanodbc.cpp:1771: 22003
[Microsoft][ODBC Driver 18 for SQL Server][SQL Server]Arithmetic overflow error converting expression to data type int.
'SELECT
COUNT() AS "number_records-count",
COUNT(DISTINCT "person_id") AS "number_subjects-count"
FROM "CDM"."visit_occurrence"'
Run rlang::last_trace() to see where the error occurred.
rlang::last_trace()
<error/rlang_error>
Error in dplyr::collect():
! Failed to collect lazy table.
Caused by error:
! nanodbc/nanodbc.cpp:1771: 22003
[Microsoft][ODBC Driver 18 for SQL Server][SQL Server]Arithmetic overflow error converting expression to data type int.
'SELECT
COUNT() AS "number_records-count",
COUNT(DISTINCT "person_id") AS "number_subjects-count"
FROM "CDM"."visit_occurrence"'
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant