Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add VLArrayCol / support variable length arrays in IsDescription #830

Open
maxnoe opened this issue Nov 26, 2020 · 4 comments
Open

Add VLArrayCol / support variable length arrays in IsDescription #830

maxnoe opened this issue Nov 26, 2020 · 4 comments

Comments

@maxnoe
Copy link
Contributor

maxnoe commented Nov 26, 2020

Support for storing variable length arrays is currently missing in pytables.

From the hdf5 documentation, this should in principle be possible.

What would be needed to imlement a VLArrayCol?

@maxnoe maxnoe changed the title Add VLArrayCol Add VLArrayCol / support variable length arrays in IsDescription Nov 26, 2020
@avalentino
Copy link
Member

Dear @maxnoe thank you for your interest in PyTables.
It seems that there is a lot of interest for supporting VL* objects in PyTables.

Personally I have not so much time to dedicate to PyTables in this period so I can't promise anything.
I suspect that the same holds for other PT developers.

Anyway if you want to submit a PR to add this feature (including a minimal testing) I will be happy to review and merge.

@HealthyPear
Copy link

@avalentino I am also interested in supporting loading from HDF5 a table like this

A B
2 [1, 2]
4 [1, 2, 3, 4]

and I think awkward-array could be used to implement this

@avalentino
Copy link
Member

Dear @HealthyPear
I don't know awkward-array.
I'm not sure to understand if it is possible to map an awkward-array into an HDF5 type in a reliable way.
Do you have already an idea about how to do it?

@HealthyPear
Copy link

Unfortunately not an expert...

I was thinking of starting this as a very complex exercise from scratch and create a new plugin for awkward for HDF5 (much like awkward-pandas), but there is also the alternative to add what's missing here (less code rewriting I guess...)

To my understanding at least one ingredient would be this
https://awkward-array.org/doc/main/user-guide/how-to-convert-buffers.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants