Stain normalization #2666

drbeh · 2021-07-28T22:36:49Z

Description

This PR implement an H&E stain normalization for histopathology images, which contains:

an H&E stain extractor using deconvolution methods from histopathology images, and
an H&E stain normalizer using the extracted stain vectors.

Unlike the previous PR (#1998), which was based on cupy, its purely based on numpy without any CUDA dependency. It expects numpy.ndarray as the input image and returns numpy.ndarray.

Status

Ready

Types of changes

Non-breaking change (fix or new feature that would not break existing functionality).
New tests added to cover the changes.
Integration tests passed locally by running ./runtests.sh -f -u --net --coverage.
Quick tests passed locally by running ./runtests.sh --quick --unittests.
In-line docstrings updated.
Documentation updated, tested make html command in the docs/ folder.

Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>

Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>

… into neha/stain-norm-2

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>

… into neha/stain-norm-2

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>

…lization

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>

wyli

looks good apart from the minor comments. @Nic-Ma could you please help confirm/approve this PR?

wyli · 2021-07-29T23:46:14Z

monai/apps/pathology/transforms/stain/array.py

+            and pseudo-max (100 - alpha percentile). Defaults to 1.
+        beta: absorbance threshold for transparent pixels. Defaults to 0.15
+        max_cref: reference maximum stain concentrations for Hematoxylin & Eosin (H&E).
+            Defaults to None.


the defauts are np.array([1.9705, 1.0308])

wyli · 2021-07-29T23:46:53Z

monai/apps/pathology/transforms/stain/array.py

+        # reshape image and calculate absorbance
+        image = image.reshape((-1, 3))
+        image = image.astype(np.float32) + 1.0
+        absorbance = -np.log(image.clip(max=self.tli) / self.tli)


do we need some validation of the input of np.log?

That's a good point! We should check if the input is intensity (0-255) and not any image or other scales. Thanks @wyli!

Now it checks the image values to be between 0 and 255.

Nic-Ma

Looks good to me except for some minor comments.
I didn't check whether the implementation matches some algorithm theory.

Thanks.

Nic-Ma · 2021-07-30T02:28:58Z

docs/source/apps.rst

+    :members:
+
+.. automodule:: monai.apps.pathology.transforms.stain.dictionary
+.. autoclass:: ExtractHEStainsD


Suggest to use ExtractHEStainsd to align with other docs, same as NormalizeHEStainsD.

Nic-Ma · 2021-07-30T02:30:52Z

monai/apps/pathology/transforms/stain/array.py

+
+        self.max_cref = max_cref
+        if self.max_cref is None:
+            self.max_cref = np.array([1.9705, 1.0308])


As @wyli commented, set default to 1.9705, 1.0308 directly.

Thanks.

Nic-Ma · 2021-07-30T02:31:46Z

monai/apps/pathology/transforms/stain/array.py

+        """Perform Stain Deconvolution and return stain matrix for the image.
+
+        Args:
+            img: uint8 RGB image to perform stain deconvolution of


Maybe of -> on or for?

Nic-Ma · 2021-07-30T02:37:49Z

monai/apps/pathology/transforms/stain/array.py

+            pseudo-max (100 - alpha percentile). Defaults to 1.
+        beta: absorbance threshold for transparent pixels. Defaults to 0.15.
+        target_he: target stain matrix. Defaults to None.
+        max_cref: reference maximum stain concentrations for Hematoxylin & Eosin (H&E).


Default values of target_he and max_cref are same as the other comment.

Nic-Ma · 2021-07-30T02:39:32Z

monai/apps/pathology/transforms/stain/dictionary.py

+        tli: float = 240,
+        alpha: float = 1,
+        beta: float = 0.15,
+        max_cref: Optional[np.ndarray] = None,


Same as the other comment.

Nic-Ma · 2021-07-30T02:39:54Z

monai/apps/pathology/transforms/stain/dictionary.py

+        tli: float = 240,
+        alpha: float = 1,
+        beta: float = 0.15,
+        target_he: Optional[np.ndarray] = None,


same comment for default value.

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>

…ain-normalization

drbeh · 2021-07-30T03:44:34Z

@wyli and @Nic-Ma
Thanks for you comments. I addressed all of them. Please let me know if I missed anything.

* added stain norm and tests Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com> * import changes Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com> * changed stain extraction tests Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com> * edited stain norm tests Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com> * convert floats to float32 Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com> * added uint8 assumption to docstring Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com> * add error case Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com> * formatting change Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com> * modify tests wrt cupy import Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com> * minor change to pass lint test Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com> * import changes Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com> * refactored classes Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com> * Restructure and rename transforms Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com> * added dict transform Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com> * Move stain_extractor to init Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com> * Exclude pathology transform tests from mini tests Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com> * Fix type checking for cupy ndarray Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com> * Include pathology transform tests Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com> * Update to cupy 9.0.0 Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com> * Remove exact version for cupy Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com> * add to docs Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com> * Organize into stain dir Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com> * Add/update init files Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com> * Transit all from cupy to numpy Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com> * Update imports Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com> * Update test cases for numpy Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com> * Rename to NormalizeHEStains and NormalizeHEStainsD Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com> * Add dictionary variant names Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com> * Fix typing and formatting Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com> * Fix docs Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com> * Update test cases Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com> * Fix clip max Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com> * Fix var typing Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com> * Fix a typing issue Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com> * Update default values, and change D to d Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com> * Update docs Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com> * Add image value check Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com> * Add test cases for negative and invalid values Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com> Co-authored-by: Neha Srivathsa <nsrivathsa@nvidia.com> Co-authored-by: nsrivathsa <81264348+nsrivathsa@users.noreply.github.com>

nsrivathsa and others added 30 commits April 12, 2021 18:17

added stain norm and tests

3e2de71

Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>

Merge branch 'master' into neha/stain-norm-2

cc6bc76

import changes

e197f9d

Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>

changed stain extraction tests

ad957fa

Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>

edited stain norm tests

00518fa

Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>

convert floats to float32

45a2ac5

Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>

added uint8 assumption to docstring

4b0f696

Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>

add error case

cb99aef

Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>

Merge branch 'master' into neha/stain-norm-2

ad29340

formatting change

3fcc366

Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>

modify tests wrt cupy import

fa95e0d

Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>

Merge branch 'master' into neha/stain-norm-2

4746f33

minor change to pass lint test

f3b2909

Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>

import changes

b80fc17

Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>

Merge branch 'master' into neha/stain-norm-2

1e36ebc

refactored classes

05ec786

Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>

Restructure and rename transforms

bcb2211

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>

Merge branch 'dev' into neha/stain-norm-2

a922bc2

added dict transform

283444e

Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>

Merge branch 'dev' into neha/stain-norm-2

dbb95d8

Move stain_extractor to init

7440a5b

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>

Exclude pathology transform tests from mini tests

f6e9b38

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>

Merge branch 'dev' into neha/stain-norm-2

26c9e5a

Fix type checking for cupy ndarray

72e0448

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>

Merge branch 'neha/stain-norm-2' of https://github.com/nsrivathsa/MONAI…

58cc4b7

… into neha/stain-norm-2

Include pathology transform tests

be88f7d

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>

Merge branch 'dev' into neha/stain-norm-2

3589326

Update to cupy 9.0.0

dd5d82e

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>

Remove exact version for cupy

128f01d

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>

Merge branch 'neha/stain-norm-2' of https://github.com/nsrivathsa/MONAI…

a462c97

… into neha/stain-norm-2

drbeh added 15 commits July 26, 2021 16:24

Add/update init files

eb411d7

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>

Transit all from cupy to numpy

9ddb4e1

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>

Update imports

24cdf11

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>

Update test cases for numpy

f26ef3a

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>

Rename to NormalizeHEStains and NormalizeHEStainsD

e7ad5a5

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>

Add dictionary variant names

7621fb4

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>

Merge branch 'dev' of github.com:Project-MONAI/MONAI into stain-norma…

f8934c3

…lization

Fix typing and formatting

e5b8c62

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>

Fix docs

b157c7c

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>

Update test cases

a050965

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>

Fix clip max

34c20c3

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>

Fix var typing

010326d

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>

Merge branch 'dev' into stain-normalization

ba52324

Fix a typing issue

d47312e

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>

Merge branch 'dev' into stain-normalization

721cba9

drbeh requested review from yiheng-wang-nv and wyli and removed request for yiheng-wang-nv July 29, 2021 14:16

wyli reviewed Jul 29, 2021

View reviewed changes

wyli requested a review from Nic-Ma July 29, 2021 23:49

Nic-Ma approved these changes Jul 30, 2021

View reviewed changes

drbeh added 5 commits July 30, 2021 03:13

Update default values, and change D to d

5f0681d

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>

Update docs

d398047

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>

Add image value check

9daead3

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>

Add test cases for negative and invalid values

bca9815

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>

Merge branch 'stain-normalization' of github.com:behxyz/MONAI into st…

4248261

…ain-normalization

wyli merged commit 30ce63d into Project-MONAI:dev Jul 30, 2021

gigony mentioned this pull request Sep 7, 2021

[FEA] Add Stain Normalization operation (in Python) rapidsai/cucim#96

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Stain normalization #2666

Stain normalization #2666

drbeh commented Jul 28, 2021

wyli left a comment

wyli Jul 29, 2021

wyli Jul 29, 2021

drbeh Jul 30, 2021

drbeh Jul 30, 2021

Nic-Ma left a comment

Nic-Ma Jul 30, 2021

Nic-Ma Jul 30, 2021

Nic-Ma Jul 30, 2021

Nic-Ma Jul 30, 2021

Nic-Ma Jul 30, 2021

Nic-Ma Jul 30, 2021

drbeh commented Jul 30, 2021

Stain normalization #2666

Stain normalization #2666

Conversation

drbeh commented Jul 28, 2021

Description

Status

Types of changes

wyli left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Nic-Ma left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

drbeh commented Jul 30, 2021