You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, the logic for converting between a MATLAB mxArray and various Arrow data structures (arrow::Table, arrow::Array, etc.) is tightly coupled and fairly tangled up with the logic specific to handling Feather files. It would be helpful to factor out these conversions into a more generic "mlarrow" conversion layer component so that it can be reused in the future for use cases other than Feather support. Furthermore, this would be helpful to enforce a cleaner separation of concerns.
It would be nice to start off with this refactoring work up front before adding support for more datatypes to the MATLAB featherread/featherwrite functions, so that we can start off with a clean base upon which to expand moving forward.
Kevin Gurney / @kevingurney:
Assigning to myself. I'll look into factoring out the MATLAB-Arrow conversions after finishing off the Feather numeric write support changes that are currently in flight here:
Todd Farmer / @toddfarmer:
This issue was last updated over 90 days ago, which may be an indication it is no longer being actively worked. To better reflect the current state, the issue is being unassigned. Please feel free to re-take assignment of the issue if it is being actively worked, or if you plan to start that work soon.
I think we can mark this as resolved now that featherread and featherwrite have been re-implemented in terms of the new internal arrow.internal.io.feather.Reader (#37163) and arrow.internal.io.feather.Writer (#37047) APIs.
The logic for converting MATLAB types to Arrow types has mostly been separated out into the arrow.array.Array subclasses at this point.
Technically, we are still casting logical and integer types to double in order to substitute null values with NaN in featherread - but, this is just to preserve compatibility with the old code, and we may change this behavior in the future.
Currently, the logic for converting between a MATLAB mxArray and various Arrow data structures (arrow::Table, arrow::Array, etc.) is tightly coupled and fairly tangled up with the logic specific to handling Feather files. It would be helpful to factor out these conversions into a more generic "mlarrow" conversion layer component so that it can be reused in the future for use cases other than Feather support. Furthermore, this would be helpful to enforce a cleaner separation of concerns.
It would be nice to start off with this refactoring work up front before adding support for more datatypes to the MATLAB featherread/featherwrite functions, so that we can start off with a clean base upon which to expand moving forward.
Reporter: Kevin Gurney / @kevingurney
Note: This issue was originally created as ARROW-3896. Please see the migration documentation for further details.
The text was updated successfully, but these errors were encountered: