Error when loading h5 file

[Yudai256]

Hi!
Although it can be read as 4D data in the Python interface, the following error occurs in the Browser:
ValueError: No 4D (or even 3D) data detected in the H5 file!

The data format is .h5 and the data shape is as follows:
(256, 256, 192, 192)

How should I address this issue?

[sezelt]

Can you share the commands you used to load the data from the Python interface, and where the data file came from (the detector? the simulation package?)

[Yudai256]

Thank you for your reply!
The data was obtained experimentally (ARINA).
I show the code below.

import py4DSTEM
from py4DSTEM.visualize import show
import matplotlib.pyplot as plt
py4DSTEM.__version__
dirpath = "./exp_data"
datacube_01 = py4DSTEM.import_file("./exp_data/data.h5")

[sezelt]

Arina data can't be opened by drag-and-drop, but you should be able to open it from File > Open Arina Data...

[Yudai256]

I tried to open data (master file) with "Open Arinta Data" but it didn't work.
I show the error message below.

ValueError: cannot reshape array of size 2415919104 into shape (16,16,192,192)

[sezelt]

What versions of py4DSTEM and py4D-browser are you using?

[Yudai256]

I have included the versions below.
py4d-browser 1.2.1
py4dstem 0.14.18

[sezelt]

When you loaded it with the Python interface via py4DSTEM.import_file("./exp_data/data.h5"), what was the shape of the data? It seems that the shape is being auto-detected incorrectly somewhere.

[Yudai256]

I show the data shap below.
Is the data of (256, 256, 192, 192) being misidentified as (16, 16, 192, 192)?

print(datacube_01.data.shape)
print(datacube_01.shape)
print(datacube_01.Rshape)
print(datacube_01.Qshape)

Dataset is uint32 but will be converted to uint16
(256, 256, 192, 192)
(256, 256, 192, 192)
(256, 256)
(192, 192)

[sezelt]

I think in a previous version of py4DSTEM Arina data was not automatically reshaped, so the GUI made an attempt to auto-detect the shape. I've made a branch of the GUI that uses py4DSTEM's shape and displays a warning when it's not detected. Could you try installing the new version and let me know if it fixes the problem? You can get it by running:

pip install git+https://github.com/sezelt/py4d-browser.git@issue31

[Yudai256]

Thank you for your kind support!
However, unfortunately, the symptoms did not improve.
The same error is occurring.

[sezelt]

Can you post the full error that you get after using the new code?

[Yudai256]

This is the error message I've got,

Traceback (most recent call last):
  File "------", line 39, in load_data_arina
    dataset.data = dataset.data.reshape(
                   ^^^^^^^^^^^^^^^^^^^^^
ValueError: cannot reshape array of size 2415919104 into shape (16,16,192,192)

However, I realized that I can read the data with the following shape:
(512, 512, 192, 192)

Does this provide any hints?

[sezelt]

Based on the error, updating to the branch with the attempted fix did not work, so you are still using the regular un-patched version. This can happen when the new version does not have a higher version number, as pip will think there is nothing to update. You can either try this to force a reinstall:

pip install --force-reinstall git+https://github.com/sezelt/py4d-browser.git@issue31

or create a new temporary environment for the test.

Try installing the patched branch this way, then see if it behaves differently.

[Yudai256]

Thanks to you, I successfully completed the loading data without any problems!
I deeply appreciate your polite and kind support.