add file size hash; make image acquisition date optional

[mtbc]

Adds the File-Size-64 hasher, adjusts Adler-32 and CRC-32 to use Guava implementations, and tweaks some import log retrieval.

To test,

• without this PR, try importing some images with various hashers set, including Adler-32 and/or CRC-32
• upgrade to OMERO5.1DEV__7
• import some of the same images with the same hashers, and also File-Size-64
  • and check that you can view the import logs
• verify that
  • the same images with the same hasher have the same hash regardless of if they were imported with or without this PR
  • for files imported with File-Size-64, their hash is a small-endian representation of their size
• for images imported after the upgrade, check that those that should have acquisition dates do (e.g., SVS) and those that shouldn't don't (e.g., PNG) (update: the latter now requires do not insert acquisition dates by default unless from metadata bioformats#1235)
  • check that null acquisition dates do not cause obvious problems: for instance, that images can still be viewed, exported, etc.

--rebased-from #2741 and fixes http://trac.openmicroscopy.org.uk/ome/ticket/12465

[mtbc]

This PR needs a breaking label at present.

[mtbc]

Okay to remove the breaking label now or should @jburel review before inclusion in merge build?

[joshmoore]

The DB changes are the ones that worry me the least, so happy to remove.

[jburel]

@mtbc, @qidane : we will need to review the XML mock Factory in bf repository where acquisition is set to be not null. not a blocker for that PR.

[jburel]

@mtbc: checking other sections of code I can think of before merging.

[jburel]

@mtbc: in ModelMockFactory, do we still want to set the acquisition time in simpleImage.
Many tests pass 0 as a parameter.

[joshmoore]

@mtbc: in discussing with @melissalinkert probably best to remove this, and in favor for making the default in Bio-Formats false. Once you've opened that PR, they can be linked together with a --depends-on.

[mtbc]

@jburel: good idea, done in #2870.

[jburel]

@mtbc, @qidane: my mistake, I checked the XMLModelMockObjects (B-F/specification), the acquisition date is not set. So nothing to change there.

[jburel]

In components/server/src/ome/logic/PixelsImpl.java
The createImage method sets the Acquisition Date to new Timestamp(new Date().getTime()));. This should be removed.

[bpindelski]

Tested with a local server@HEAD of develop which later got upgraded to snoopy's merge build.

• CRC-32 and Adler-32 checksums are the same for images imported pre- and post-DB upgrade,
• file size hasher indeed stores the images size in little-endian order,
• acquisition date appears for SVS but doesn't for DV or png.

I can't see any issues with the diff, so I'd say this is good to merge.

[jburel]

@mtbc: tested with some other images after yesterday stress.
Both images imported against octopus and trout. Same date displayed i.e. date from the files.

[jburel]

I will do some final tests tomorrow am.

[jburel]

@mtbc, @dominikl: The fact that the acquisition date can now be null will lead to NPE in the recent search code.
I am currently fixing a time issue related to search in dev_5_0 so the problem can be fixed in another PR since this is already large.

[jburel]

unless anybody wants to have a final check, we should probably merge the PR.

[mtbc]

It's been a week and listed in stand-up a few times: I think that everybody who might have wanted to probably already has. (-:

[sbesson]

See #2892. Integration tests using fake image with and without acquisition dates pass. Ready to merge.