Add no StatsInfo option to importer

[chris-allan]

This PR adds a --no_stats_info option to OMERO.importer to allow for the disabling of minima and maxima calculation when not present as part of the Bio-Formats reader metadata. This is of particular benefit where the data volume being imported is extremely high (10s of GB) and the person importing the data is aware of the tradeoffs that apply. Under such conditions calculating global minima and maxima can take hours. We have already applied a similar strategy to images of the digital pathology domain.

The significant bits populated as part of the import process are now being used to inform the default dynamic range of the rendering engine if such metadata is available. Furthermore, certain areas of the code that did not populate the bit width when creating a PixelsType have been fixed. At least one area where the pixels type name was being compared incorrectly has also been fixed.

Unit and integration tests cover the semantic implementation outlined here. Functional testing will have to be discussed.

/cc @emilroz

[jburel]

To import images: use
bin/omero import -- --no_stats_info fileName Thanks @chris-allan for tip

[jburel]

both insight and web work fine.
Possible to view the image.
Small adjustments required in clients e.g. min/max grey out if no statsinfo
not to be done in this PR

Removing breaking

[jburel]

The only case with the --no-stats option, stats info will be created is when a pyramid is generated.
Discussed with @chris-allan,
This is expected and it comes at no cost since we need to go through all the pixels

[sbesson]

Tested on a large SPIM dataset with --no_stats_info at import. Import got stuck before min/max calculation while parsing the pixel data to set the Pixels hash. I will try to work from this branch and optionally disable this unnecessary data reading step if the --no_stats_info option is passed.

[sbesson]

Thanks @chris-allan. Merging this and I will open a follow-up PR for additional pixel checksum disabling.

[sbesson]

--no-rebase

[mtbc]

Is there (or is it worth) a card somewhere for being able to cause calculation of this stuff in the background for already imported images?

[jburel]

There is no card, the only please at the moment we use the stats info is at the rendering engine level.
Not having the stats info, means that the full pixels range will be used. Then the user will adjust the settings.
Probably not worth a card now, we can review depending on feedback.

[jburel]

There is no card, the only place at the moment we use the stats info is at the rendering engine level.
Not having the stats info, means that the full pixels range will be used. Then the user will adjust the settings.
Probably not worth a card now, we can review depending on feedback.

[sbesson]

@mtbc: this point was partially discussed in ome/omero-documentation#1152 while adding the limitation. I share @jburel's thoughts, we need to drive this only if there is a concrete well-defined need from a particular community.