Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Setting data alignment at File creation #2034

Closed
hmaarrfk opened this issue Jan 8, 2022 · 4 comments · Fixed by #2040
Closed

Setting data alignment at File creation #2034

hmaarrfk opened this issue Jan 8, 2022 · 4 comments · Fixed by #2040

Comments

@hmaarrfk
Copy link
Contributor

hmaarrfk commented Jan 8, 2022

I'm hoping to add data alignment strategies to certain files. This can be achieved by adding parameters to make_fapl in:

https://github.com/h5py/h5py/blob/master/h5py/_hl/files.py#L111

Using set_alignment. We would need to add 2 parameters, the threshold, and the alignment interval.

One would need to add the lines

def make_fapl(...., threshold=1, alignment_interval=1):
    fapl.set_alignment(threshold, alignment_interval)

and provide additional parameters to the File constructor.

Would this be an acceptable PR to make?

Thank you!

References: https://support.hdfgroup.org/HDF5/doc/UG/FmSource/08_TheFile_favicon_test.html

To assist reproducing bugs, please include the following:

  • Operating System (e.g. Windows 10, MacOS 10.11, Ubuntu 16.04.2 LTS, CentOS 7): Linux
  • Python version (e.g. 2.7, 3.5): 3.9
  • Where Python was acquired (e.g. system Python on MacOS or Linux, Anaconda on
    Windows): conda-forge Linux
  • h5py version (e.g. 2.6): 3.6.0
  • HDF5 version (e.g. 1.8.17): 1.12.1
@takluyver
Copy link
Member

That seems reasonable enough to me - we've already gone for lots of keyword parameters for File(). @tacaswell, are you OK with adding a couple more?

I'd probably call them alignment_threshold (to make it clear what threshold it is) and alignment (following the name HDF5 uses).

@hmaarrfk
Copy link
Contributor Author

There might be some other configuration that would be desirable for performance reasons, cache size, and whatnot.

Do you wan tto make the fapl and fack overwrite able instead? not very high level, but gives the most control

@tacaswell
Copy link
Member

So long as they are un-coupled (that is the user can correctly set just a sub-set) I'm 👍 on adding as many extra keyword-only arguements to File as we need to expose everything in facl.

If there are a bunch of the inputs that only make sense when the file is created (rather than just opened), what do people think of adding some classmethod constructors instead of adding to the __init__ methods? It might require a bit of jiggery-pokery, but I think we could make it work without having to close and then re-open the file...

@hmaarrfk
Copy link
Contributor Author

@tacaswell great.

This is an access parameter, so I believe it is valid both at creation time and at open time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants