Possible and useful to implement Apache Arrow in IPLD?

[timcash]

Apache Arrow is popular in the data science space as it reduces time to load into memory, would it be possible to integrate the format into the Dag? Any tips on where to get started with IPLD to take a shot at it? https://arrow.apache.org/overview/

[rvagg]

That'd probably depend on how you imagine using it with IPLD, could you explain a workflow that might connect Arrow and IPLD @timcash?

[timcash]

Good question, still learning about IPLD but I was thinking pulling fields out of files in Arrow format. E.g. "/CID/session_id/0" would get the first "session_id" out of the file with that CID.