Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

issue with VDS and self referencing VirtualSource #4080

Open
payno opened this issue Mar 7, 2024 · 2 comments
Open

issue with VDS and self referencing VirtualSource #4080

payno opened this issue Mar 7, 2024 · 2 comments
Assignees
Labels
Component - C Library Core C library issues (usually in the src directory) Priority - 2. Medium ⏹ It would be nice to have this in the next release Type - Bug Please report security issues to help@hdfgroup.org instead of creating an issue on GitHub UNCONFIRMED New issues are unconfirmed until a maintainer can duplicate them

Comments

@payno
Copy link

payno commented Mar 7, 2024

Describe the bug
It looks like there is some consistency or bug regarding 'self-referencing' VirtualSource (so if the source dataset is contained in the file containing also the virtual dataset - VDS)

Expected behavior
If a "file_name" contains a dataset referenced in VDS contained in the same file then if the virtual source path is "." or ".{file_name}" should have the same behavior

Platform (please complete the following information)
tested (only) with:

  • HDF5version: 1.14.3
  • OS and version: Ubuntu 20.04.6 LTS
  • default HDF5 version provided by h5py wheel

Additional context
Here is some code to reproduce (using h5py):
first assertion works as expected, second one failed (using '.{file_path}' instead of '.')

import numpy
import h5py

print("** Write test datasets")
data = numpy.linspace(1, 10, 100, dtype=numpy.float32).reshape((10, 10))


with h5py.File("test.h5", mode="w") as h5f:
    h5f["/data"] = data
    vsource_1 = h5py.VirtualSource(".", "/data", shape=(10, 10))
    layout = h5py.VirtualLayout(shape=(10, 10), dtype="f4")
    layout[:] = vsource_1
    h5f.create_virtual_dataset("/vds_ok", layout, fillvalue=-5)

    vsource_1 = h5py.VirtualSource("./test.h5", "/data", shape=(10, 10))
    layout = h5py.VirtualLayout(shape=(10, 10), dtype="f4")
    layout[:] = vsource_1
    h5f.create_virtual_dataset("/vds_failed", layout, fillvalue=-5)


print("** Read with locking")
with h5py.File("test.h5", mode="r", locking=True) as h5f:
    vds_data = h5f["/vds_ok"][()]
    assert numpy.array_equal(data, vds_data)
    vds_data = h5f["/vds_failed"][()]
    assert numpy.array_equal(data, vds_data)


print("** Read without locking")
with h5py.File("test.h5", mode="r", locking=False) as h5f:
    print("  - VDS OK")
    vds_data = h5f["/vds_ok"][()]
    assert numpy.array_equal(data, vds_data)
    print("  - vds_failed")
    vds_data = h5f["/vds_failed"][()]
    assert numpy.array_equal(data, vds_data)
@glennsong09 glennsong09 added Priority - 2. Medium ⏹ It would be nice to have this in the next release Component - C Library Core C library issues (usually in the src directory) Type - Bug Please report security issues to help@hdfgroup.org instead of creating an issue on GitHub UNCONFIRMED New issues are unconfirmed until a maintainer can duplicate them labels Mar 7, 2024
@fortnern
Copy link
Member

fortnern commented Mar 7, 2024

Does this fail both with and without file locking?

@payno
Copy link
Author

payno commented Mar 8, 2024

With file locking it works. Without it fails.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Component - C Library Core C library issues (usually in the src directory) Priority - 2. Medium ⏹ It would be nice to have this in the next release Type - Bug Please report security issues to help@hdfgroup.org instead of creating an issue on GitHub UNCONFIRMED New issues are unconfirmed until a maintainer can duplicate them
Projects
None yet
Development

No branches or pull requests

3 participants