Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix error message when reading hspy compressed using blosc #2760

Conversation

ericpre
Copy link
Member

@ericpre ericpre commented Jun 2, 2021

When reading a hspy file saved using blosc compression, an unclear error message is raised if hdf5plugin hasn't been imported previously - see silx-kit/hdf5plugin#69 (comment).

Progress of the PR

  • Try to import hdf5plugin when reading hspy file to avoid unclear error message,
  • add an changelog entry in the upcoming_changes folder (see upcoming_changes/README.rst),
  • ready for review.

Minimal example of the bug fix or the new feature

Create a signal and save it using blosc compression

import hdf5plugin
import hyperspy.api as hs
import numpy as np

s = hs.signals.Signal1D(np.random.random(10*10).reshape((10, 10)))
s.save('test.hspy', compression=hdf5plugin.Blosc(cname='blosclz', clevel=9, shuffle=hdf5plugin.Blosc.SHUFFLE))

Try to open it in a new python process

import hyperspy.api as hs

s = hs.load('test.hspy')

It gives the following error message:

ERROR:hyperspy.io:If this file format is supported, please report this error to the HyperSpy developers.
Traceback (most recent call last):

  File "<ipython-input-3-bed9dafb25ee>", line 1, in <module>
    s = hs.load('test.hspy')

  File "/home/eric/Dev/hyperspy/hyperspy/io.py", line 423, in load
    objects = [load_single_file(filename, lazy=lazy, **kwds) for filename in filenames]

  File "/home/eric/Dev/hyperspy/hyperspy/io.py", line 423, in <listcomp>
    objects = [load_single_file(filename, lazy=lazy, **kwds) for filename in filenames]

  File "/home/eric/Dev/hyperspy/hyperspy/io.py", line 476, in load_single_file
    return load_with_reader(filename=filename, reader=reader, **kwds)

  File "/home/eric/Dev/hyperspy/hyperspy/io.py", line 496, in load_with_reader
    file_data_list = reader.file_reader(filename, **kwds)

  File "/home/eric/Dev/hyperspy/hyperspy/io_plugins/hspy.py", line 180, in file_reader
    exp = hdfgroup2signaldict(exg, lazy)

  File "/home/eric/Dev/hyperspy/hyperspy/io_plugins/hspy.py", line 237, in hdfgroup2signaldict
    data = np.asanyarray(data)

  File "/opt/miniconda3/lib/python3.8/site-packages/numpy/core/_asarray.py", line 171, in asanyarray
    return array(a, dtype, copy=False, order=order, subok=True)

  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper

  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper

  File "/opt/miniconda3/lib/python3.8/site-packages/h5py/_hl/dataset.py", line 1012, in __array__
    self.read_direct(arr)

  File "/opt/miniconda3/lib/python3.8/site-packages/h5py/_hl/dataset.py", line 973, in read_direct
    self.id.read(mspace, fspace, dest, dxpl=self._dxpl)

  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper

  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper

  File "h5py/h5d.pyx", line 192, in h5py.h5d.DatasetID.read

  File "h5py/_proxy.pyx", line 112, in h5py._proxy.dset_rw

OSError: Can't read data (can't open directory: /opt/miniconda3/lib/hdf5/plugin)

…r message when reading file compressed with blosc and hdf5plugin haven't already imported
@ericpre ericpre added this to the v1.6.3 milestone Jun 2, 2021
@codecov
Copy link

codecov bot commented Jun 2, 2021

Codecov Report

Merging #2760 (522be0b) into RELEASE_next_patch (6d184cd) will increase coverage by 0.00%.
The diff coverage is 100.00%.

Impacted file tree graph

@@                 Coverage Diff                 @@
##           RELEASE_next_patch    #2760   +/-   ##
===================================================
  Coverage               77.30%   77.30%           
===================================================
  Files                     202      202           
  Lines                   30032    30036    +4     
  Branches                 6561     6561           
===================================================
+ Hits                    23216    23220    +4     
  Misses                   5068     5068           
  Partials                 1748     1748           
Impacted Files Coverage Δ
hyperspy/io_plugins/hspy.py 69.01% <100.00%> (+0.26%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6d184cd...522be0b. Read the comment docs.

@jlaehne jlaehne merged commit 07baffe into hyperspy:RELEASE_next_patch Jun 3, 2021
@ericpre ericpre deleted the fix_reading_hspy_compression_blosc branch June 3, 2021 08:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants