Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

c/driver/postgresql: Handle non-COPY path results where individual string data would be larger than 2GB #2064

Open
paleolimbot opened this issue Aug 6, 2024 · 0 comments
Labels
Type: enhancement New feature or request

Comments

@paleolimbot
Copy link
Member

What feature or improvement would you like to see?

In the COPY reader we go to quite some lengths to ensure that we don't overflow the 2GB limit for the Arrow string type:

na_res = AppendRowAndFetchNext(&error);
if (na_res == EOVERFLOW) {
// The result would be too big to return if we appended the row. When EOVERFLOW is
// returned, the copy reader leaves the output in a valid state. The data is left in
// pg_buf_/data_ and will attempt to be appended on the next call to GetNext()
return BuildOutput(out, &error);
}

This was not implemented in the PR that enabled converting PQresult objects to arrays (#2029). While it is less likely that one of these queries will overflow (given that it is opt-in), we have the infrastructure to handle it and should!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Type: enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant