Version 1 - 116 improve output data formatting

[emlynjdavies]

This is a PR to progress the main branch to Version 1.

This is a summary of where i ended up with the interpretation of the steps dictionary from a toml configuration file, and saving it to nc (instead of h5), and including a steps string that is possible to re-evaluate to re-run the analysis:
notebooks/single-image-stats.ipynb

Completed issues contained in this PR

• fixes for holograms - Initial step requires a filename - maybe better to take the general raw files folder and read the first file from that #133
• fixes pixel size should be included in output files #96
• fixes converted steps_string that is saved as metadata may lose information on functions objects passed to step classes #65
• fixes make cli command that updates an existing config.toml file #139
• fixes Check for consistency across documentation and examples #134

---

Holo test assertion change:

Number of particles expected in this test is now 56 instead of 42. This change is likely due to different processing settings used in the test. See pyopia/tests/test_pipeline.py

---

Known issue with Mac M2 dependencies. For now, please use the https://github.com/SINTEF/pyopia/tree/mac-m branch if you don't have an intel mac

[emlynjdavies]

This method will have problems when it comes to handling steps that contain inputs that are not available in a config file e.g.

'create background': pyopia.background.CreateBackground(bgfiles, pyopia.instrument.silcam.load_image)

@animmosmith would it be acceptable to insist that pipeline classes only take inputs of strings or floats that don't require computation (i.e. can be put in a config file)? this would mean a bit of refactoring?

Or are there other alternatives?

[animmosmith]

This method will have problems when it comes to handling steps that contain inputs that are not available in a config file e.g.

'create background': pyopia.background.CreateBackground(bgfiles, pyopia.instrument.silcam.load_image)

@animmosmith would it be acceptable to insist that pipeline classes only take inputs of strings or floats that don't require computation (i.e. can be put in a config file)? this would mean a bit of refactoring?

Or are there other alternatives?

I think reworking things to make it work with a config file is worth it. I guess with the upgrade to 3.10 we can make use of match-case constructs to enable easy switching between difference processing functions?

[emlynjdavies]

Getting back to this now, after the summer...

In the case of the background creation, we currently do:

bgfiles = files[:average_window] # provide a list of background files used to create the static background

I am thinking that we will need to change this to a folder name, which the background creator uses to find bgfiles. so it would then have the folder path and average_window as inputs in the config.

Does that sound OK @animmosmith ?

[animmosmith]

Getting back to this now, after the summer...

In the case of the background creation, we currently do:

bgfiles = files[:average_window] # provide a list of background files used to create the static background

I am thinking that we will need to change this to a folder name, which the background creator uses to find bgfiles. so it would then have the folder path and average_window as inputs in the config.

Does that sound OK @animmosmith ?

Yes, I think that would be easy to explain to end users if using the static background option - an additional option to the background creator tool could be either how many of the files in the folder to use (equally distributed) or what the stride length should be if sub-sampling the files in the (perhaps raw data) folder for speed.