Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Apache Arrow methods #32

Open
1 of 4 tasks
randyzwitch opened this issue Oct 30, 2018 · 4 comments
Open
1 of 4 tasks

Implement Apache Arrow methods #32

randyzwitch opened this issue Oct 30, 2018 · 4 comments
Labels

Comments

@randyzwitch
Copy link
Contributor

randyzwitch commented Oct 30, 2018

Implement remaining load_table methods, with an eye on removing unneeded methods (such as row-wise loading instead of columnar)

load_table_binary(conn::OmniSciConnection, table_name::String, rows::Vector{TRow})

  • load_table_binary_columnar(conn::OmniSciConnection, table_name::String, cols::Vector{TColumn})
  • load_table_binary_arrow(conn::OmniSciConnection, table_name::String, arrow_stream::Vector{UInt8})
  • sql_execute_df
  • sql_execute_gdf
@randyzwitch
Copy link
Contributor Author

Removing load_table_binary from focus, as there is an error that I currently don't understand. It appears that pymapd doesn't implement the method, which might give a good indication its not needed

@randyzwitch randyzwitch changed the title Implement remaining load_table methods Implement load_table_binary_arrow method Nov 1, 2018
@randyzwitch
Copy link
Contributor Author

Remaining method needs Arrow support

@randyzwitch randyzwitch added apache arrow enhancement New feature or request labels Nov 1, 2018
@randyzwitch
Copy link
Contributor Author

This isn't just creating Arrow vectors (from Arrow.jl), but a stream/recordbatch. Might need to use ArrowGlib.jl

@randyzwitch randyzwitch changed the title Implement load_table_binary_arrow method Implement Apache Arrow methods Dec 11, 2018
@randyzwitch
Copy link
Contributor Author

WIth JuliaData/FlatBuffers.jl#24, another piece for Arrow support getting closer. With Flatbuffer flatc, should be able to move closer to a native Arrow implementation to read recordbatches

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant