-
Notifications
You must be signed in to change notification settings - Fork 128
Description
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
DuckDB has a replacement scan feature in the Python API that allows it to directly access Python variables that are Arrow-based.
This would mean that you wouldn't necessarily need to perform a register_dataframe
step in advance; you could just query a variable directly. This would also be temporary; it wouldn't persist the dataframe as a registered object in the session.
Describe the solution you'd like
If a table name is not found among the registered
Is there some way to implement an order of operations for finding tables with a certain name? I.e. inject some code so that if a table x
isn't found already-registered, then it can evaluate whether the Python variable x
is Arrow-like?
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
I could implement the Python side of this, but I don't know the DataFusion API well enough to know where to find the hooks or how to connect to them.