New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Impl Polars cursor #436
Comments
polars cursor would be a godsend! |
Polars uses Arrow as its memory representation, so, as I understand it, supporting Polars in PyAthena is mostly just a syntactic shortcut, right? Polars' documentation for the
So except for that note about unsupported types, the following code should have basically no overhead already today:
I actually tried out PyAthena → Arrow → Polars in this fashion the other day, so I can at least confirm this is functional (i.e. it will populate a Polars DataFrame that works, I didn't verify anything about copying or performance overheads) |
Hi, may I ask what version of pyarrow are you using ? I have an error with version OperationalError: When reading information for key 'test/670672a1-dab2-4635-ba3b-1c6a16dc0b6f.csv' in bucket '{BucketName}': AWS Error NETWORK_CONNECTION during HeadObject operation: curlCode: 28, Timeout was reached or if another solution comes to your mind to explain this error. Thank you |
This was many months ago, I no longer recall what version I was using... but your error is to all appearances a network connectivity problem, says so right there in the message |
Yes, I totally agree, but it's cryptic to me since it's working with another cursor (like pandasCursor for example)
|
An error occurred when I ran the test in the local environment. 🤔 |
https://www.pola.rs/
https://pypi.org/project/polars/
https://pola-rs.github.io/polars/py-polars/html/reference/
The text was updated successfully, but these errors were encountered: