Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add reader for hamamatsu .img format #87

Merged
merged 19 commits into from Aug 17, 2023
Merged

Conversation

pietsjoh
Copy link
Contributor

@pietsjoh pietsjoh commented Feb 10, 2023

Add support for reading Hamamatsu's .img file format.

Progress of the PR

  • basic parsing of the file
  • map metadata
  • add tests,
  • update docstring (if appropriate),
  • add an changelog entry in the upcoming_changes folder (see upcoming_changes/README.rst),
  • Check formatting changelog entry in the readthedocs doc build of this PR (link in github checks)
  • cover different acquisition modes with testfiles
  • reproduce plots
  • update tests after problem fix
  • ready for review.

Minimal example of the bug fix or the new feature

from rsciio.hamatsu import file_reader
file_reader("file.img", reader="Hamamatsu")

@codecov
Copy link

codecov bot commented Feb 10, 2023

Codecov Report

Patch coverage: 85.54% and project coverage change: +0.18% 🎉

Comparison is base (9d0f04f) 83.75% compared to head (73cd7d6) 83.94%.
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #87      +/-   ##
==========================================
+ Coverage   83.75%   83.94%   +0.18%     
==========================================
  Files          74       76       +2     
  Lines        9800    10132     +332     
  Branches     2163     2210      +47     
==========================================
+ Hits         8208     8505     +297     
- Misses       1097     1114      +17     
- Partials      495      513      +18     
Files Changed Coverage Δ
rsciio/hamamatsu/_api.py 85.36% <85.36%> (ø)
rsciio/hamamatsu/__init__.py 100.00% <100.00%> (ø)

... and 13 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

rsciio/hamamatsu/_api.py Fixed Show fixed Hide fixed
rsciio/hamamatsu/_api.py Fixed Show fixed Hide fixed
rsciio/hamamatsu/_api.py Fixed Show fixed Hide fixed
rsciio/hamamatsu/_api.py Fixed Show fixed Hide fixed
rsciio/hamamatsu/_api.py Fixed Show fixed Hide fixed
rsciio/hamamatsu/_api.py Fixed Show fixed Hide fixed
@jlaehne
Copy link
Contributor

jlaehne commented Jun 2, 2023

@Attolight-NTappy and @gkusch - I know you are happy using the tiff import of Hamamatsu Streak images, but maybe you could also have a look at this reader for the Hamamatsu format?

@gkusch
Copy link

gkusch commented Jun 2, 2023

I will try to export a Streak in the Format next time I am at the microscope and compare with .tiff Currently measurements done with the Attolight Software controlling the Streak will be autosaved as .tiff so not super relevant for us in general. Are there any inherent advantages of this format vs tif or is it more a matter of providing choice?

@jlaehne
Copy link
Contributor

jlaehne commented Jun 2, 2023

I will try to export a Streak in the Format next time I am at the microscope and compare with .tiff Currently measurements done with the Attolight Software controlling the Streak will be autosaved as .tiff so not super relevant for us in general. Are there any inherent advantages of this format vs tif or is it more a matter of providing choice?

According to the Hamamatsu specs, img supports 32 bit data, while their tif export supports only 16 bit data.
See https://github.com/bigphatjan/itex2ascii/blob/master/HiPicFileFormat.pdf

Both readers do read in the original_metadata. The one for img already maps some information to metadata.

But I'm curious what your comparison would give.

@pietsjoh
Copy link
Contributor Author

While reproducing plots, I discovered some problems. I didn't put much thought into reading the data block, because I thought it was just a block of continuous data. Hopefully, changing the endianness fixes the problem. But the plot still slightly differs compared to the original from a matlab script (which performs multiple steps). I will further check this and try to reproduce it exactly.

@pietsjoh pietsjoh force-pushed the hamamatsu branch 2 times, most recently from e5f34f9 to 479bd1d Compare August 3, 2023 10:27
@pietsjoh
Copy link
Contributor Author

pietsjoh commented Aug 4, 2023

While reproducing plots, I discovered some problems. I didn't put much thought into reading the data block, because I thought it was just a block of continuous data. Hopefully, changing the endianness fixes the problem. But the plot still slightly differs compared to the original from a matlab script (which performs multiple steps). I will further check this and try to reproduce it exactly.

The problem is fixed now.

@jlaehne
Copy link
Contributor

jlaehne commented Aug 7, 2023

Indeed, the data looks good now. I compared a real dataset for which I have both the .img and .tif files.

@jlaehne
Copy link
Contributor

jlaehne commented Aug 11, 2023

According to the Hamamatsu specs, img supports 32 bit data, while their tif export supports only 16 bit data.
See https://github.com/bigphatjan/itex2ascii/blob/master/HiPicFileFormat.pdf

We should include this format documentation in the documentation under File Specifications for later reference.

@@ -0,0 +1,21 @@
.. _hamamatsu-format:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add to index.rst

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is added in docs/supported_formats/index.rst.

docs/supported_formats/hamamatsu.rst Outdated Show resolved Hide resolved
rsciio/hamamatsu/specifications.yaml Outdated Show resolved Hide resolved
rsciio/hamamatsu/specifications.yaml Show resolved Hide resolved
docs/supported_formats/hamamatsu.rst Outdated Show resolved Hide resolved
----------
%s
%s

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add use_uniform_signal_axis parameter, e.g. by copying the docstring from jobin-yvon (or by adding it to _docstrings)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I copied it for now and will do a follow up PR, where I add this to _docstrings and move the signal axis fit to utils.

@jlaehne jlaehne merged commit 0077aeb into hyperspy:main Aug 17, 2023
30 of 32 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants