You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Curious if anyone is actively working on this? Would be a nice addition to the python bindings, I wouldn't mind taking a shot at it, but I probably would have a hard unless heavily guided.
This is something that would be absolutely fantastic. DuckDB enables this kind of a workflow and we've found it really valuable. The ibis API for Datafusion pretends to do it too... but in reality the .collect() call buffers the whole query result into memory so saves nothing... It'd be nice to improve this!
mesejo
added a commit
to mesejo/arrow-datafusion-python
that referenced
this issue
Mar 6, 2024
arrow-rs now has zero-copy to PyArrow for RecordBatchReaders.
We could use this implement the execute_stream and execute_stream_partitioned methods for DataFrames similar to the Rust DataFrame.
The text was updated successfully, but these errors were encountered: