File size and channel names

[smith6jt-cop]

Hello!
The new version is working great performance-wise compared with the previous version. Just a couple of issues hopefully are easy fixes. Running the latest Docker image on Windows 10 Ubuntu 22.04.3 LTS (GNU/Linux 5.15.133.1-microsoft-standard-WSL2 x86_64). I tested with two 4-channel cycles from multiplex run and using the following script, the file size was 8,897,156KB. Resaving as .tif from FIJI the size was 1,480,200KB. Opening the .ome.tiff in FIJI there were two series although I used pyramid=False when writing. Is there a way to ensure a minimal file size and only one series writes? Beside the file size/pyramiding, the other minor issue is the channel_name_dict is being ignored. The channel names being written are sequential integers followed by the original input folder name (not original file names). The input tiffs did have channel names including dapi for channel 0, but there were not recognized.

import valis
from valis import valtils
from valis import registration
from valis import slide_io
from valis.micro_rigid_registrar import MicroRigidRegistrar

import time
import os
import numpy as np

valis.slide_io.init_jvm(jar=None, mem_gb=50)

slide_src_dir = "/home/smith6jt/VALIS_in"
results_dst_dir = "/home/smith6jt/VALIS_out"
merged_slide_dst_f = "/home/smith6jt/VALIS_out/20_6_SP_CC2C.ome.tiff"
path_to_registrar = "/home/smith6jt/VALIS_out/VALIS_in/data/VALIS_in_registrar.pickle"
micro_reg_fraction = 0.25

registrar = registration.Valis(slide_src_dir, results_dst_dir, micro_rigid_registrar_cls=MicroRigidRegistrar)
rigid_registrar, non_rigid_registrar, error_df = registrar.register()

Calculate what max_non_rigid_registration_dim_px needs to be to do non-rigid registration on an image that is 25% full resolution.

img_dims = np.array([slide_obj.slide_dimensions_wh[0] for slide_obj in registrar.slide_dict.values()])
min_max_size = np.min([np.max(d) for d in img_dims])
img_areas = [np.multiply(d) for d in img_dims]
max_img_w, max_img_h = tuple(img_dims[np.argmax(img_areas)])
micro_reg_size = np.floor(min_max_sizemicro_reg_fraction).astype(int)

Perform high resolution non-rigid registration using 25% full resolution

micro_reg, micro_error = registrar.register_micro(max_non_rigid_registration_dim_px=micro_reg_size)

channel_name_dict = {"cyc01.tif" : ["DAPI", "Blank1a", "Blank1b", "Blank1c"],
"cyc02.tif" : ["DAPI", "CD31", "CD8", "Empty2c"]}

"cyc03.tif" : ["DAPI", "CD20", "Ki67", "CD3e"],

"cyc04.tif" : ["DAPI", "SMActin", "Podoplanin", "CD68"],

"cyc05.tif" : ["DAPI", "PanCK", "CD21", "CD4"],

"cyc06.tif" : ["DAPI", "Lyve1", "CD45RO", "CD11c"],

"cyc07.tif" : ["DAPI", "CD35", "ECAD", "CD107a"],

"cyc08.tif" : ["DAPI", "CD34", "CD44", "HLADR"],

"cyc09.tif" : ["DAPI", "Empty9a", "FoxP3", "CD163"],

"cyc10.tif" : ["DAPI", "Empty10a", "CollagenIV", "Vimentin"],

"cyc11.tif" : ["DAPI", "Empty11a", "CD15", "CD45"],

"cyc12.tif" : ["DAPI", "Empty12a", "CD5", "CD1c"],

"cyc13.tif" : ["DAPI", "Blank13a", "Blank13b", "Blank13c"]}

"cyc014_Z="+pos+".tif" : ["DAPI", "Empty14", "CD45RA", "Empty14b"],

"cyc015_Z="+pos+".tif" : ["DAPI", "Blank15a", "Blank15b", "Blank15c"]}

registrar = registration.load_registrar(path_to_registrar)
merged_img, channel_names, ome_xml =
registrar.warp_and_merge_slides(
merged_slide_dst_f,
channel_name_dict=channel_name_dict,
pyramid=False,
drop_duplicates=False,
crop="overlap")

registration.kill_jvm() # Kill the JVM

These are the images that ended up working. At first, I saved merged channels after deconvolution and extended depth of field processing using a FIJI macro, but got AttributeError: 'NoneType' object has no attribute 'doubleValue' and UnboundLocalError: local variable 'slide_reader_cls' referenced before assignment. To fix this, I resaved after splitting channels and saving as a stack.

https://drive.google.com/drive/folders/19SfyorAqbGGMRFeoE-v50dx8hOreFQAe?usp=sharing

Thank you for all your hard work!

[smith6jt-cop]

Not sure if this will help, but I resaved the images as ome.tif in FIJI using latest BIoformats exporter and ran the same script:

docker run -v "$HOME:$HOME" cdgatenbee/valis-wsi python3 /home/smith6jt/VALIS_reg_files/VALIS_merge.py

/usr/local/src/.venv/lib/python3.10/site-packages/PIL/Image.py:3176: DecompressionBombWarning: Image size (94499532 pixels) exceeds limit of 89478485 pixels, could be decompression bomb DOS attack.
warnings.warn(
/usr/local/src/.venv/lib/python3.10/site-packages/PIL/Image.py:3176: DecompressionBombWarning: Image size (94509018 pixels) exceeds limit of 89478485 pixels, could be decompression bomb DOS attack.
warnings.warn(
/usr/local/src/.venv/lib/python3.10/site-packages/PIL/Image.py:3176: DecompressionBombWarning: Image size (94479608 pixels) exceeds limit of 89478485 pixels, could be decompression bomb DOS attack.
warnings.warn(
/usr/local/src/.venv/lib/python3.10/site-packages/PIL/Image.py:3176: DecompressionBombWarning: Image size (94509494 pixels) exceeds limit of 89478485 pixels, could be decompression bomb DOS attack.
warnings.warn(
/usr/local/src/.venv/lib/python3.10/site-packages/PIL/Image.py:3176: DecompressionBombWarning: Image size (94499055 pixels) exceeds limit of 89478485 pixels, could be decompression bomb DOS attack.
warnings.warn(
/usr/local/src/.venv/lib/python3.10/site-packages/PIL/Image.py:3176: DecompressionBombWarning: Image size (94508540 pixels) exceeds limit of 89478485 pixels, could be decompression bomb DOS attack.
warnings.warn(
/usr/local/src/.venv/lib/python3.10/site-packages/PIL/Image.py:3176: DecompressionBombWarning: Image size (94538907 pixels) exceeds limit of 89478485 pixels, could be decompression bomb DOS attack.
warnings.warn(
/usr/local/src/valis/valtils.py:24: UserWarning: Can't find slide file associated with 21_12_SP_CC3_A_LED2_EDF
warnings.warn(warning_msg, warning_type)

==== Converting images

some non-RGB channel names were None or not provided. Renamed channels are: ['C1', 'C2', 'C3', 'C4']
some non-RGB channel names were None or not provided. Renamed channels are: ['C1', 'C2', 'C3', 'C4']
Converting images: 0%| | 0/2 [00:00<?, ?image/s]
/usr/local/src/valis/valtils.py:24: UserWarning: unable to call VipsForeignLoadTiffFile
tiff2vips: no SUBIFD tag

warnings.warn(warning_msg, warning_type)
Traceback (most recent call last):
File "/home/smith6jt/VALIS_reg_files/VALIS_merge.py", line 23, in
Traceback (most recent call last):
File "/usr/local/src/valis/registration.py", line 4189, in register
self.convert_imgs(series=self.series, reader_cls=reader_cls, reader_dict=reader_dict)
File "/usr/local/src/valis/registration.py", line 2375, in convert_imgs
vips_img = reader.slide2vips(level=level)
File "/usr/local/src/valis/slide_io.py", line 1700, in slide2vips
vips_slide = self._slide2vips_ome_one_series(level=level, *args, **kwargs)
File "/usr/local/src/valis/slide_io.py", line 1657, in _slide2vips_ome_one_series
toilet_roll = pyvips.Image.new_from_file(self.src_f, n=-1, subifd=level-1)
File "/usr/local/src/.venv/lib/python3.10/site-packages/pyvips/vimage.py", line 352, in new_from_file
return pyvips.Operation.call(name, filename,
File "/usr/local/src/.venv/lib/python3.10/site-packages/pyvips/voperation.py", line 305, in call
raise Error('unable to call {0}'.format(operation_name))
pyvips.error.Error: unable to call VipsForeignLoadTiffFile
tiff2vips: no SUBIFD tag

JVM has been killed. If this was due to an error, then a new Python session will need to be started
min_max_size = np.min([np.max(d) for d in img_dims])
File "/usr/local/src/.venv/lib/python3.10/site-packages/numpy/core/fromnumeric.py", line 2953, in min
return _wrapreduction(a, np.minimum, 'min', axis, None, out,
File "/usr/local/src/.venv/lib/python3.10/site-packages/numpy/core/fromnumeric.py", line 88, in _wrapreduction
return ufunc.reduce(obj, axis, dtype, out, **passkwargs)
ValueError: zero-size array to reduction operation minimum which has no identity

[cdgatenbee]

Thanks @smith6jt-cop for reporting this and sharing the images. I'll use them to see if I can solve the issues in your first post. For the second post, the error is most likely because I had assumed that all ome.tiffs would have SUBIFD tags, but it looks like that isn't always true. Anyhow, I can try to generate a similar ome.tiff without a subifd to recreate the issue, but it would be great if you could share the ome.tiff so that I can double check the fix. There are a few other things I need to work on today, so I probably will have to wait until next week to dig into this. I'll keep you updated though.

Best,
-Chandler

[smith6jt-cop]

I found that this is not specifically a VALIS issue, but rather another example of how FIJI, QuPath, and Python don't always play nicely together. The last step of my image processing pipeline before VALIS has the images going through the Clij2 FIJI plugin to perform extended depth of field which collapses the 3D images to 2D.
Then, after VALIS the images do have the channel names when opening in QuPath even though they are not shown in FIJI. Moreover, resaving VALIS-processed images in FIJI as tif and the channel names are not visible in QuPath. I found that resaving the images in a jupyter notebook (Python) immediately after VALIS processing with:

for i in range(1, 14):
c=str(i)
image=("C:\Users\smith6jt\20_8_SP_CC2B(new)_EDF\Converted\cyc0"+c.zfill(2)+".tif")
im=skimage.io.imread(image)
im=np.asarray(im)
skimage.io.imsave("C:\Users\smith6jt\20_8_SP_CC2B(new)_EDF\Converted\converted_cyc0"+c.zfill(2)+".tif", im)

not only solves the problem I reported above related to large file size (now only 16GB), but also solves the problem I was having with VALIS throwing a an "UnboundLocalError: local variable 'reader' referenced before assignment" which may be related to #84. Overall, I believe I need to educate myself on how metadata is stored and managed in each of these platforms.

Edit: One issue that remains which is VALIS-related is that pyramidal images are still being written with the script above that includes pyramid=False in merge_and_save_slides.

[smith6jt-cop]

Here is a link to the ome.tiff https://drive.google.com/file/d/1fg7G4iFtn0HPTyqJmfjOqaYPDx9AiJAi/view?usp=sharing
Also, opening with AICSImage gives:

Attempted file (c:/Users/smith6jt/20_06_N1_R2_EDF_forVALIS/20_06_N1_R2.ome.tiff) load with reader: aicsimageio.readers.ome_tiff_reader.OmeTiffReader failed with error: No module named 'xmlschema'

[cdgatenbee]

Thanks for all of this, @smith6jt-cop! I may not be able to get to this until Friday, but I'll keep you updated.

Best,
-Chandler

[smith6jt-cop]

Thanks for the update. It's odd now I'm opening with pyvips and cannot get more than the first channel of the 40 channel ome.tiff from VALIS to open although all channels are available with scikit-image, pillow, microimage, and matplotlib. All the channel information is stored under the metadata 'image-description' but is not accessible for some reason, except by QuPath. Thanks, Justin Smith, PhD Dept of Pathology, Immunology, and Laboratory Medicine UF Diabetes Institute College of Medicine 1275 Center Drive, 5th Floor Gainesville, FL 32610 From: Chandler Gatenbee ***@***.***> Sent: Wednesday, February 14, 2024 8:43 AM To: MathOnco/valis ***@***.***> Cc: Smith,Justin A ***@***.***>; Mention ***@***.***> Subject: Re: [MathOnco/valis] File size and channel names (Issue #101) [External Email] Thanks for all of this, @smith6jt-cop<https://github.com/smith6jt-cop>! I may not be able to get to this until Friday, but I'll keep you updated. Best, -Chandler - Reply to this email directly, view it on GitHub<#101 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ARQJN42PFWY3HARZDCFGGKLYTS5MPAVCNFSM6AAAAABDAAJ27SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNBTG44TIOBTGI>. You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>>

[cdgatenbee]

Hi @smith6jt-cop,

I realized that valis continued saving an image pyramid even when pyramid=False, because subifd=True in pyvips.Image.tiffsave. I think we had figured this detail out previously, but it somehow didn't get in the last update. I've added that fix, as well as a few other things to handle non-pyramid ome-tiffs, including: making sure MetaData.slide_dimensions only has 1 value, instead of the same dimension repeated once for each channel; having a tile size that's smaller than the image (Bioformats and/or ImageJ seems to set the tile size of non-pyramid image to be whatever the largest dimension is). With these changes, valis saves your images as tiled non-pyramid ome-tiffs that are readable by Bio-formats (tested in QuPath and valis), and valis seems to be able to open your 40 channel image (including channel names) without issue.

Regarding pyvips only reading the first channel, it may be because reading ome-tiffs isn't straightforward using pyvips, as you're expected piece the image back together. So if you just use pyvips.Image.new_from_file, you'll get

import pyvips
pyvips_img = pyvips.Image.new_from_file(src_f)
# <pyvips.Image 9985x9504 ushort, 1 bands, grey16>

But if you use the ome2vips function to piece the image together (based on valis.slide_io.VipsSlideReader._slide2vips_ome_one_series), you'll get the expected image

def ome2vips(src_f, level, *args, **kwargs):
        """Use pyvips to read an ome.tiff image that has only 1 series

        Pyvips throws an error when trying to read other series
        because they may have a different shape than the 1st one
        https://github.com/libvips/pyvips/issues/262

        Parameters
        -----------
        level : int
            Pyramid level

        Returns
        -------
        vips_slide : pyvips.Image

        """

        toilet_roll = pyvips.Image.new_from_file(src_f, n=-1, subifd=level-1)
        page = pyvips.Image.new_from_file(src_f, n=1, subifd=level-1, access='random')
        if page.interpretation == "srgb":
            vips_slide = page
        else:
            page_height = page.height
            pages = [toilet_roll.crop(0, y, toilet_roll.width, page_height) for
                     y in range(0, toilet_roll.height, page_height)]

            vips_slide = pages[0].bandjoin(pages[1:])
            if vips_slide.bands == 1:
                vips_slide = vips_slide.copy(interpretation="b-w")
            else:
                vips_slide = vips_slide.copy(interpretation="multiband")

        return vips_slide


ome_vips = ome2vips(src_f, level=0)
# <pyvips.Image 9985x9504 ushort, 40 bands, multiband>

Finally, I don't know if you've tested the various compression methods already, but I wanted to see if a different compression method could help you reduce your file size while also maintaining the correct datatype (here uint16). The default compression is "lzw" and it creates huge files because it's lossless, but setting compression="jp2k" and Q=90 will significantly reduce your file size. For example, using the 2 images you shared, the lzw compressed image is 9.3GB (pyramid image), but only 1.01GB (non-pyramid) or 1.24GB (pyramid) with jp2k compression. Unexpectedly, I also found that the "lzw" compressed image doesn't seem to display correctly in QuPath, but the "jp2k" compressed one does. If you're interested, you can try these different settings when you call Valis.warp_and_merge_slides:

merged_img, channel_names, ome_xml = registrar.warp_and_merge_slides(
    dst_f=merged_slide_dst_f,
    channel_name_dict=channel_name_dict,
    pyramid=pyramid,
    drop_duplicates=False,
    compression="jp2k",
    Q=90,
    crop="overlap")

At the moment, I don't think there are many other outstanding issues or additions, so I may be able to push these changes as part of an update soon.

Best,
-Chandler

[smith6jt-cop]

Hello! Any update on when the changes might be pushed to Docker? Thanks!

[cdgatenbee]

Hi @smith6jt-cop,
So sorry for the delay. This update has ended up including some bigger changes that I'd anticipated, so it's taking a bit longer than expected. It's mostly compete, but there remain a few final bugs to work out. I'll be sure to let you know when it's ready though.

Best,
-Chandler

[smith6jt-cop]

No problem! Keep up the good work! Thanks, Justin Smith, PhD Dept of Pathology, Immunology, and Laboratory Medicine UF Diabetes Institute College of Medicine 1275 Center Drive, 5th Floor Gainesville, FL 32610 From: Chandler Gatenbee ***@***.***> Sent: Tuesday, April 16, 2024 10:20 AM To: MathOnco/valis ***@***.***> Cc: Smith,Justin A ***@***.***>; Mention ***@***.***> Subject: Re: [MathOnco/valis] File size and channel names (Issue #101) [External Email] Hi @smith6jt-cop<https://github.com/smith6jt-cop>, So sorry for the delay. This update has ended up including some bigger changes that I'd anticipated, so it's taking a bit longer than expected. It's mostly compete, but there remain a few final bugs to work out. I'll be sure to let you know when it's ready though. Best, -Chandler - Reply to this email directly, view it on GitHub<#101 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ARQJN44ODNGQQD7NXRMJZP3Y5UXSFAVCNFSM6AAAAABDAAJ27SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANJZGIYTIMBQGQ>. You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>>