Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

adding support for generating batches of arrow tables #133

Closed
thenomemac opened this issue Oct 6, 2017 · 2 comments
Closed

adding support for generating batches of arrow tables #133

thenomemac opened this issue Oct 6, 2017 · 2 comments
Labels
done feature

Comments

@thenomemac
Copy link

I'm writing a CLI using turbdodbc's arrow support to dump the arrow arrays to disk as parquet files... I'm interested in contributed support to batch "K" rows of large tables this will allow for tables larger than memory. Would there be an easy path forward for adding this feature. Maybe I'm missing something but seems like there should be a fetcharrowbatch() method?
@MathMagique
Copy link
Member

Hi there! That sounds like a useful project of yours, and the feature you suggest makes perfect sense. There should be a fetcharrowbatches() similar to the existing fetchnumpybatches() that you can use for your tool.

@MathMagique MathMagique mentioned this issue Oct 20, 2017
Closed
@mariusvniekerk mariusvniekerk mentioned this issue Jun 22, 2018
Merged
@MathMagique
Copy link
Member

Done in #172

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
done feature
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@MathMagique @thenomemac