Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update WSI foreground segmentation algorithm #456

Conversation

nkaenzig
Copy link
Collaborator

@nkaenzig nkaenzig commented May 17, 2024

Closes #455

I used a simplified version of the segmentation algorithm from CLAM. CLAM on top of what is implemented here filters the approximate contours of the detected foreground objects based on an area threshold as a post processing step. We don't need this step here, because our patching logic anyways applies another foreground threshold filter to the sampled patches, so very small foreground areas in the mask most likely won't yield patches anyways.

Comparison

  • "grayscale-threshold" is what we've been using so far (for datasets such as panda which have a clean white background this worked reasonably well, but it breaks for datasets such as camelyon16 where this is not the case)
  • mb stands for median blur applied to saturation channel in HSV image
  • otsu thresholding applied to saturation channel doesn't seem to perform better than using a fixed threshold
  • I would recommend disabling the fill_holes in our default setup, as filled holes can result in patches sampled from within holes which might not contain relevant tissue data.
image

Avg Runtime per WSI [s]

grayscale_threshold 0.089027
hsv 0.071794
hsv + mb 0.138696
hsv + mb + holes 0.140928
hsv + mb + otsu 0.138782
hsv + mb + otsu + holes 0.143519

  • median blur is computationally the most expensive step, but it generates cleaner and smoother masks. 138ms per WSI is still fast enough, so I recommend to enable it.

@nkaenzig nkaenzig self-assigned this May 17, 2024
@nkaenzig nkaenzig changed the title mask unit tests added Update WSI foreground segmentation algorithm May 17, 2024
@nkaenzig nkaenzig marked this pull request as ready for review May 17, 2024 12:43
Copy link
Collaborator

@ioangatop ioangatop left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great 🚀

@nkaenzig nkaenzig merged commit e3e5671 into 360-aggregated-feature-support-wsi-level-tasks May 21, 2024
5 checks passed
@nkaenzig nkaenzig deleted the 455-improve-foreground-mask-algorithm branch May 21, 2024 06:48
roman807 added a commit that referenced this pull request Jun 24, 2024
* Add base class for WSIs & `openslide` implementation (#365)

* Add WSI dataset classes (#368)

* Add baseline `panda` workflow (#373)

* add panda config

* adjust batch size

* addressed comments

* addressed comments

* addressed comments

* Add support for grid sampling to `WsiDataset` (#377)

* Replaced cached_property in WsiDataset by LRU cache (#388)

* Updated `EmbeddingsWriter` to support multi-embedding file outputs (#384)

* Simple baseline sampler for foreground patches (#394)

* add foreground kaggle

* added random foreground sampler kaggle

* refactored sampler

* moved get_mask

* typo

* refactor sampler

* refactor sampler

* addressed comments

* addressed comments

* Fixed linting in WSI feature branch (#407)

* added openslide-python to all dependencies

* Fix input batch class name (#414)

* Add lower bound for wsi resolution level during mask generation (#412)

* Move sampler logic to `samplers` module and add unit tests (#420)

* Add `WsiClassificationDataset` (#429)

* Retrieve MPP from WSIs  (#432)

* add mpp conversion

* formatting

* addressed comments

* addressed comments

* addressed comments

* formatting

* formatting

* formatting

* updated panda config (#437)

* update WSI foreground segmentation algorithm (#456)

* Add `PANDA` dataset class (#430)

* fixed panda (#463)

* Update `EmbeddingsWriter` to store tensors as lists (#465)

* add tiffslide backend (#468)

* added panda cli unit tests (#470)

* move wsi initialization to setup method of `MultiWsiDataset` (#472)

* 459 add camelyon16 slide level task (#476)

* added panda dataset class

* clean up

* remove samples with noisy labels

* clean up table in dataset readme

* added function for stratified splits

* added unit tests

* cleanup

* addressed comments

* fixed issue with resource download

* validation fix

* updated readme

* added to mkdocs

* added image_dir to exception print

* updated root path in yaml config

* added panda to datasets overview table in docs

* added md5 hash for downloaded resources

* update init

* added camelyon16

* added camelyon16

* updated camelyon16 class

* added tests and config

* formatting

* formatting

* formatting

* formatting

* added test files

* formatting

* lint

* added target transforms

* formatting

* fixed dataset

* addressed comments

* addressed comments

* fix test

* fix test

* fixed test

* addressed comments

* updated loss

* fix annotations

* lint

---------

Co-authored-by: Nicolas Kaenzig <nkaenzig@gmail.com>

* 475 define slide level evaluation protocol (#511)

* updated configs

* adjust patience

* addressed comments

* fixed typo

* remove prefetch factor

* update 360-aggregated-feature before PR to main (#527)

* Updated developer guide (#418)

* Update `TotalSegmentator2D` dataset to fetch all the slices (#416)

* Move metrics to CPU when using single device (#446)

* Remove total segmentator classification dataset (#450)

* updated eva logo (#454)

* updated eva logo

* renamed files

* Update actions/checkout digest to a5ac7e5 (#458)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Add configuration logger (#466)

* Update `README` with paper citation (#474)

* update docs (#482)

* Update img shields of README (#480)

* Fix `torch` and `jsonargparse` versions (#483)

* update depedencies

* update

* bump micro version (#486)

* update config links (#487)

* Update paper citation format (#489)

* Update the vision dataset return types to `tv_tensors` (#478)

* Refactor embeddings writer (#461)

* fixed phikon configs (#493)

* Refactor embeddings datasets (#495)

* Add doc tests and minor fixes (#492)

* support setting download as env-variable (#514)

* updated confis and doc

* typo

* update datasets

* fixed types

* src/eva/core/callbacks/writers/embeddings/base.py

* formatting

* types

---------

Co-authored-by: Nicolas Känzig <36882833+nkaenzig@users.noreply.github.com>
Co-authored-by: ioangatop <johngatop@gmail.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Updated documentation with new datasets and leaderboard (#531)

* updated layout

* updated layout

* addressed comment

* 532 update leaderboard results with slide level tasks (#538)

* updated docs

* update leaderboard

* update docs and links

* updated configs (#539)

* updated leaderboard (#543)

---------

Co-authored-by: Nicolas Känzig <36882833+nkaenzig@users.noreply.github.com>
Co-authored-by: Nicolas Kaenzig <nkaenzig@gmail.com>
Co-authored-by: ioangatop <johngatop@gmail.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants