Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Graceful Error Handling in Particle Patches #1609

Closed
ax3l opened this issue Apr 15, 2024 · 3 comments
Closed

Graceful Error Handling in Particle Patches #1609

ax3l opened this issue Apr 15, 2024 · 3 comments

Comments

@ax3l
Copy link
Member

ax3l commented Apr 15, 2024

Describe the bug
We recently introduced more graceful error handling when we encounter incomplete data.
I found another data set, which we use in openPMD-updater, that only contains structure but no real data in it.

In this data set, particlePatches cause read errors in openPMD-ls.

To Reproduce
Download the file structure.h5 from openPMD/openPMD-example-datasets#22 / https://github.com/openPMD/openPMD-example-datasets/blob/draft/structure.h5

$ openpmd-ls structure.h5 
openPMD series: structure
openPMD standard: 1.1.0
openPMD extensions: 1

data author: Axel Huebl <a.huebl@hzdr.de>
data created: 2018-02-20 12:42:57 +0100
data backend: HDF5
generating machine: ax3l
generating software: openPMD Example Script (version: 1.1.0.1)
generating software dependencies: python@2.7.13;numpy@1.13.3;hdf5@1.8.18;h5py@2.7.1

number of iterations: 1 (groupBased)
HDF5-DIAG: Error detected in HDF5 (1.12.2) thread 0:
  #000: H5A.c line 454 in H5Aopen(): unable to open attribute: 'unitDimension'
    major: Attribute
    minor: Can't open object
  #001: H5VLcallback.c line 1091 in H5VL_attr_open(): attribute open failed
    major: Virtual Object Layer
    minor: Can't open object
  #002: H5VLcallback.c line 1058 in H5VL__attr_open(): attribute open failed
    major: Virtual Object Layer
    minor: Can't open object
  #003: H5VLnative_attr.c line 124 in H5VL__native_attr_open(): unable to open attribute: 'unitDimension'
    major: Attribute
    minor: Can't open object
  #004: H5Aint.c line 422 in H5A__open(): unable to load attribute info from object header for attribute: 'unitDimension'
    major: Attribute
    minor: Can't open object
  #005: H5Oattribute.c line 494 in H5O__attr_open_by_name(): can't locate attribute: 'unitDimension'
    major: Attribute
    minor: Object not found
  all iterations: [AbstractIOHandlerImpl] IO Task READ_ATT failed with exception. Clearing IO queue and passing on the exception.
Cannot read patch record 'extent' due to read error and will skip it:Read Error in backend HDF5
Object type:	Attribute
Error type:	NotFound
Further description:	[HDF5] Internal error: Failed to open HDF5 attribute 'unitDimension' (/data/0/particles/electrons/particlePatches/extent/) during attribute read
HDF5-DIAG: Error detected in HDF5 (1.12.2) thread 0:
  #000: H5A.c line 454 in H5Aopen(): unable to open attribute: 'unitDimension'
    major: Attribute
    minor: Can't open object
  #001: H5VLcallback.c line 1091 in H5VL_attr_open(): attribute open failed
    major: Virtual Object Layer
    minor: Can't open object
  #002: H5VLcallback.c line 1058 in H5VL__attr_open(): attribute open failed
    major: Virtual Object Layer
    minor: Can't open object
  #003: H5VLnative_attr.c line 124 in H5VL__native_attr_open(): unable to open attribute: 'unitDimension'
    major: Attribute
    minor: Can't open object
  #004: H5Aint.c line 422 in H5A__open(): unable to load attribute info from object header for attribute: 'unitDimension'
    major: Attribute
    minor: Can't open object
  #005: H5Oattribute.c line 494 in H5O__attr_open_by_name(): can't locate attribute: 'unitDimension'
    major: Attribute
    minor: Object not found
[AbstractIOHandlerImpl] IO Task READ_ATT failed with exception. Clearing IO queue and passing on the exception.
Cannot read patch record 'offset' due to read error and will skip it:Read Error in backend HDF5
Object type:	Attribute
Error type:	NotFound
Further description:	[HDF5] Internal error: Failed to open HDF5 attribute 'unitDimension' (/data/0/particles/electrons/particlePatches/offset/) during attribute read
0 

number of meshes: 3
  all meshes:
    B
    E
    rho

number of particle species: 1
  all particle species:
    electrons

Expected behavior
Should as it does for the other missing data gracefully skip

Software Environment

  • version of openPMD-api: 0.15.2
  • installed openPMD-api via: conda-forge
  • operating system: Ubuntu 22.04
  • machine: no
  • name and version of Python implementation: 3.11.6
  • version of HDF5: 1.14.2
  • version of ADIOS2: N/A
  • name and version of MPI: N/A

Additional context
@s-sajid-ali and me are working on 2.0 updates for https://github.com/openPMD/openPMD-updater and moved the little example file in it to https://github.com/openPMD/openPMD-example-datasets via openPMD/openPMD-example-datasets#22

@ax3l
Copy link
Member Author

ax3l commented Apr 16, 2024

X-ref: #1439 pt. 3

@ax3l
Copy link
Member Author

ax3l commented Apr 17, 2024

The problem is actually that particle patches are not openPMD records https://github.com/openPMD/openPMD-standard/blob/latest/STANDARD.md#sub-group-for-each-particle-species Update: they are records, in a sub-group.

But the API seems to currently parse it as such, looking for unitDimension on extent, offset, etc.

@ax3l
Copy link
Member Author

ax3l commented Apr 17, 2024

Fix coming for particle patches example from openPMD-validator via openPMD/openPMD-validator#75 and openPMD/openPMD-example-datasets#23

@ax3l ax3l closed this as completed Apr 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant