Support PreciPoint VMIC format

[bgilbert]

Apparently it's roughly Deep Zoom inside ZIP.

[Markus-PP]

I am ready to work on this. Has anyone else given the matter some thoughts yet ? Obviously it requires a ZIP library. After some tests, I concluded that libzip does what we need. Are there any objections against including libzip ? Will there be other slides with images in containers supported by libzip ?

[bgilbert]

Awesome, thanks for looking into this! When writing your code, please ensure that you do not use any information obtained under a non-disclosure agreement. If you use NDA-protected material, we will not be able to merge your changes.

libzip would work, but wouldn't be my first choice. libgsf will read both ZIP and OLE files, which might be useful someday for Zeiss CZI. If you do use libzip, note that version 1.0 is not yet available in many supported Linux distributions. I'm not currently aware of any other primary formats that use ZIP containers, but I'd suggest making an openslide-decode-gsf.c (or -decode-zip.c) anyway.

Do you have sample VMIC files?

[bgilbert]

Does VMIC use ZIP64?

• libgsf < 1.14.31 (Debian through jessie, Ubuntu through vivid, RHEL 6 and 7) will choke on ZIP64.
• libzip < 0.11 (Debian through wheezy, Ubuntu through trusty, RHEL 6 and 7) doesn't support ZIP64, and from a quick glance at the 0.10.1 source, may also choke.

[Markus-PP]

Thanks for fast reply ! I have approval and support of the PreciPoint team so i don't see any legal issue here. I do have sample files, and I know ZIP64 is actually required. Thanks for hinting that older versions don't support it - I did test runs with libzip 1.1.2 (the newest) which works like charm on VMIC, will have to check if libzip 0.11.2 and libgsf do the job, too. Also planning to set up an openslide-vendor-deepzoom.c for generalizing purposes. Will probably have more questions but not at this point. Much studying is still required.

[My distribution, OpenSuSE 13.2, has libzip 0.11.2 out-of-the-box, updateable from repo to 1.1.2; and libgsf 1.14.30, updateable to 1.14.33]

[bgilbert]

If older versions of a library simply reject ZIP64, that's fine. If they crash or behave incorrectly (as libgsf does) we will need to conditionally enable ZIP support.

Are you able to share the sample files under CC0? We strongly prefer that new drivers come with freely-redistributable samples. (Indeed, we can't create automated test cases without them.)

Some background on the NDA stuff: we have encountered vendors who have offered to provide documentation for an OpenSlide driver, but only under restrictions which are unworkable for an open-source project. Therefore, to avoid problems, we've been planning to require contributors to certify that they have the right to contribute the code they are submitting under OpenSlide's license. It's the same process used in Linux kernel development: each Git commit message will need to include a Signed-off-by: line (as produced by git commit -s), which certifies that the change is submitted in accordance with the Developer Certificate of Origin. I haven't had a chance yet to document this policy on the OpenSlide website, but it will apply to your contributions. Please let me know if you have any questions, and thanks for your understanding.

[Markus-PP]

PreciPoint team have agreed on providing some VMIC- example files for public domain.

https://drive.google.com/file/d/0BwWPfDJXmk2gbm5ROU5TZUNxVms/view?usp=sharing
https://drive.google.com/file/d/0BwWPfDJXmk2gbUdsU1V2TGE4eHM/view?usp=sharing
https://drive.google.com/file/d/0BwWPfDJXmk2gcG5NME5zNGgwTnM/view?usp=sharing

I have written some code which is in experimental stage and somewhat working, although not ready for inclusion in the library. Shall I discuss it here or rather in the mailing list instead ?

[bgilbert]

Thanks for the slide files! Note that public domain isn't legally valid in all countries. Would it be possible for the samples to be licensed under CC0? (CC0 is essentially public domain plus a permissive license for places where public domain doesn't exist.)

If your code isn't ready for a pull request, I'd suggest discussing it on the mailing list.

[Markus-PP]

Yes, of course they can be licensed under CC0, as well, since the PreciPoint team ensured they are devoid of third-party rights. Is perhaps any official statement needed ?

A commit is under way, but before doing so i'm going to send a reply on the mailing list.

[Markus-PP]

Repository https://github.com/Markus-PP/openslide-vmic has a functional driver.

[bgilbert]

@Markus-PP For now, if you could post an official statement in this thread (or send it to me privately), I could add the samples to openslide-testdata. That would allow VMIC tests to be added to the test suite.

Eventually we'll have a web form for collecting CC0 signoffs, and at that point I'll ask you to complete it.

[Markus-PP]

I hereby confirm that the PreciPoint team agrees to release the above mentionend 3 slides under CC0 and that, to the best of my knowledge, they are devoid of any third party rights.

[bgilbert]

Thank you very much!

I just noticed that each sample includes a copy of OpenSeadragon, which can't be licensed under CC0 because it is subject to third-party copyright. OpenSeadragon is BSD-licensed and the license information is properly included in the slide files, so we can redistribute the samples, but I'll need to mark them as a mix of CC0 and BSD.

In each of the inner ZIP files, there is a small number of members with a .jpg.bmp extension. What is the purpose of those members?

[bgilbert]

Also:

• Can you easily provide any additional descriptive metadata for each of the files? (Stain, type of tissue, etc.) It would be useful for the textual description of each slide (example).
• Do you or PreciPoint want to be credited for the samples, and if so, how should we credit you? (See above example.)
• Does your software care about the names of slide files? In other words, if I rename the files to <something>.vmic, will that cause any problems?

[Markus-PP]

To the first question: I guess, at the moment it's probably best to remove the seadragon references, because they are not needed for any of the viewer applications. The original goal was to provide the customer a way to easily unpack and view the slide without extra software. The bogus jpg.bmp files, if I remember correctly, were created during testing and they have no purpose. I'll talk with the team next week to know for sure, Then, I can also fetch the textual description, together with the seadragon-free slides.
Any credits should go to PreciPoint, if possible.
The VMIC driver as implemented here does not care about any part of the file name, nor does the viewer application, so yes they can be renamed to anything.

[bgilbert]

Great, I'll credit PreciPoint and rename the files.

We should avoid modifying the samples after they are written by your software; otherwise they won't be accurate samples. If current versions of your software no longer embed OpenSeadragon, then I'd be in favor of generating some new scans. Otherwise, let's use unmodified files and just list both licenses. (Mixed licensing is not uncommon in open-source projects.)