[BUG] Compression scheme 33003 tile decoding is not implemented. #17
Comments
|
Hello @OlivierDehaene ! Compression scheme 33003 is actually JPEG2000 which is used in Aperio SVS format. cuCIM currently supports tiled RGB TIFF image(Generic TIFF or Philips TIFF) with JPEG or Deflate compressed and doesn't support SVS format yet. The tentative plan is specified in the following link (I would move the content to Github wiki page later): If cuCIM encounters TIFF (SVS is a variant of TIFF format) file that is not JPEG/Deflate compressed RGB format, it just delegates its process to libtiff's TIFFRGBAImageGet() method and it seems that warning/error are not properly handled. Let me fix the issue by raising an error until cuCIM supports SVS format properly. Thank you for the report! We are also eager to support various image formats so stay tuned! |
SVS format is not supported for now but we loosen the restriction by delegating its process to libtiff's TIFFRGBAImageGet() method. cuCIM assumes that only one image with Subfile Type(=0) is available in the image file but apparently, some SVS files have also only one Subfile Type with zero so bypassing the check. This patch explicitly checks SVS format and compression methods and raises an exception if the given file doesn't match the criteria. Once SVS format is supported, those changes need to be updated to not raise exceptions for SVS files.
|
Thank you for your answer @gigony!
Amazing! I have been looking for a better alternative to OpenSlide for a while now and if cuCIM starts to support Aperio that would be great :)
Will we be able to disable the cache when doing random tile reads? We generally disable the OpenSlide one when training. |
I thought cuCIM can allocate some memory for cache by default and the user can increase the cache size programmatically. I assumed it wouldn't be a problem if we use a cache memory per 'parent process' and cache memory is shared among subprocesses (whereas OpenSlide seems to allocate cache memory per file). However, you're right. It is better to have an option to disable cache at all as cache wouldn't much help when reading random partial images from a huge image pool during training. Now I am thinking of allocating cache memory only when requested :) Thank you so much for your valuable feedback! |
SVS format is not supported for now but we loosen the restriction by delegating its process to libtiff's TIFFRGBAImageGet() method. cuCIM assumes that only one image with Subfile Type(=0) is available in the image file but apparently, some SVS files have also only one Subfile Type with zero so bypassing the check. This patch explicitly checks SVS format and compression methods and raises an exception if the given file doesn't match the criteria. Once SVS format is supported, those changes need to be updated to not raise exceptions for SVS files. Resolves: #17 Authors: - Gigon Bae (https://github.com/gigony) Approvers: - Benjamin Zaitlen (https://github.com/quasiben) - https://github.com/jakirkham URL: #19
|
Some packages should be pushed to |
Due to #17 ([BUG] Compression scheme 33003 tile decoding is not implemented), #19 (Check compression method used in the image) was merged so cuCIM has been handling only particular formats (only jpeg/deflate-compressed image). If the input TIFF image has no-compressed RAW tile image, cuCIM showed the following error message: > RuntimeError: This format (compression: 1, sample_per_pixel: 1, planar_config: 1, photometric: 1) is not supported yet!. https://github.com/Project-MONAI/MONAI/pull/2987/checks?check_run_id=3667530814#step:7:15641 This patch is to support raw RGB tiled TIFF with fast-path, re-allowing non-compressed image with slow-path (in case the input is not tiled RGB image). This supports Project-MONAI/MONAI#2987. Authors: - Gigon Bae (https://github.com/gigony) Approvers: - Gregory R. Lee (https://github.com/grlee77) - https://github.com/jakirkham URL: #108
- Update Jpeg decoder to consider colorspace of the JPEG image
- We need to set a proper color space from TIFF metadata to the decoder
- Update tjDecompress2() method call with jpeg_decode_buffer() that modified
tjDecompress2() implementation
- Add `jpeg_color_space_` to IFD Class.
- Move additional includes to their own cmake files
- Add Jpeg2k decoder with OpenJpeg
- Added fast color conversion logic with pre-calculated table
- Add LZW decoder with libtiff's implementation
- Take part of the code to provide only LZW decoder part
- Add `rows_per_strip_` and `predictor_` attributers to IFD class, to support
LZW
- Refactor TIFF:resolve_vendor_format() to support Aperio SVS metadata
- Support LZW compressed image (with multi strips) for associated image.
Addresses rapidsai#17
- Update Jpeg decoder to consider colorspace of the JPEG image
- We need to set a proper color space from TIFF metadata to the decoder
- Update tjDecompress2() method call with jpeg_decode_buffer() that modified
tjDecompress2() implementation
- Add `jpeg_color_space_` to IFD Class.
- Move additional includes to their own cmake files
- Add Jpeg2k decoder with OpenJpeg
- Added fast color conversion logic with pre-calculated table
- Add LZW decoder with libtiff's implementation
- Take part of the code to provide only LZW decoder part
- Add `rows_per_strip_` and `predictor_` attributers to IFD class, to support
LZW
- Refactor TIFF:resolve_vendor_format() to support Aperio SVS metadata
- Support LZW compressed image (with multi strips) for associated image.
Addresses rapidsai#17
- Update Jpeg decoder to consider colorspace of the JPEG image
- We need to set a proper color space from TIFF metadata to the decoder
- Update tjDecompress2() method call with jpeg_decode_buffer() that modified
tjDecompress2() implementation
- Add `jpeg_color_space_` to IFD Class.
- Move additional includes to their own cmake files
- Add Jpeg2k decoder with OpenJpeg
- Added fast color conversion logic with pre-calculated table
- Add LZW decoder with libtiff's implementation
- Take part of the code to provide only LZW decoder part
- Add `rows_per_strip_` and `predictor_` attributers to IFD class, to support
LZW
- Refactor TIFF:resolve_vendor_format() to support Aperio SVS metadata
- Support LZW compressed image (with multi strips) for associated image.
Addresses rapidsai#17
- Update Jpeg decoder to consider colorspace of the JPEG image
- We need to set a proper color space from TIFF metadata to the decoder
- Update tjDecompress2() method call with jpeg_decode_buffer() that modified
tjDecompress2() implementation
- Add `jpeg_color_space_` to IFD Class.
- Move additional includes to their own cmake files
- Add Jpeg2k decoder with OpenJpeg
- Added fast color conversion logic with pre-calculated table
- Add LZW decoder with libtiff's implementation
- Take part of the code to provide only LZW decoder part
- Add `rows_per_strip_` and `predictor_` attributers to IFD class, to support
LZW
- Refactor TIFF:resolve_vendor_format() to support Aperio SVS metadata
- Support LZW compressed image (with multi strips) for associated image.
Addresses rapidsai#17
- Update Jpeg decoder to consider colorspace of the JPEG image
- We need to set a proper color space from TIFF metadata to the decoder
- Update tjDecompress2() method call with jpeg_decode_buffer() that modified
tjDecompress2() implementation
- Add `jpeg_color_space_` to IFD Class.
- Move additional includes to their own cmake files
- Add Jpeg2k decoder with OpenJpeg
- Added fast color conversion logic with pre-calculated table
- Add LZW decoder with libtiff's implementation
- Take part of the code to provide only LZW decoder part
- Add `rows_per_strip_` and `predictor_` attributers to IFD class, to support
LZW
- Refactor TIFF:resolve_vendor_format() to support Aperio SVS metadata
- Support LZW compressed image (with multi strips) for associated image.
Addresses rapidsai#17
- Update Jpeg decoder to consider colorspace of the JPEG image
- We need to set a proper color space from TIFF metadata to the decoder
- Update tjDecompress2() method call with jpeg_decode_buffer() that modified
tjDecompress2() implementation
- Add `jpeg_color_space_` to IFD Class.
- Move additional includes to their own cmake files
- Add Jpeg2k decoder with OpenJpeg
- Added fast color conversion logic with pre-calculated table
- Add LZW decoder with libtiff's implementation
- Take part of the code to provide only LZW decoder part
- Add `rows_per_strip_` and `predictor_` attributers to IFD class, to support
LZW
- Refactor TIFF:resolve_vendor_format() to support Aperio SVS metadata
- Support LZW compressed image (with multi strips) for associated image.
Addresses rapidsai#17
**Add tif_lzw.c from libtiff as it is** Copy tif_lzw.c file from libtiff for porting lzw decoder. The code is from the following link: https://gitlab.com/libtiff/libtiff/-/blob/8546f7ee994eacff0a563918096f16e0a6078fa2/libtiff/tif_lzw.c , which is after v4.3.0 **Support Aperio SVS with CPU LZW and jpeg2k decoder** - Update Jpeg decoder to consider colorspace of the JPEG image - We need to set a proper color space from TIFF metadata to the decoder - Update tjDecompress2() method call with jpeg_decode_buffer() that modified tjDecompress2() implementation - Add `jpeg_color_space_` to IFD Class. - Move additional includes to their own CMake files - Add Jpeg2k decoder with OpenJpeg - Added fast color conversion logic with pre-calculated table - Add LZW decoder with libtiff's implementation - Take part of the code to provide only LZW decoder part - Add `rows_per_strip_` and `predictor_` attributes to IFD class, to support LZW - Refactor TIFF:resolve_vendor_format() to support Aperio SVS metadata - Support LZW compressed image (with multi strips) for the associated image. Addresses #17    Authors: - Gigon Bae (https://github.com/gigony) Approvers: - https://github.com/jakirkham URL: #141
Describe the bug
For Aperio svs files, I run into a cryptic error:
/slide.svs: Compression scheme 33003 tile decoding is not implemented.However, cuCMI still loads a tile and doesn't raise but the color are wrong or it is entirely black. See the following grid of tiles:
Are all OpenSlide vendors supported? Is there a list of all supported vendors somewhere? I could not find it.
Steps/Code to reproduce bug
The text was updated successfully, but these errors were encountered: