add region growing algorithm #13

oberonia78 · 2023-06-24T02:42:35Z

This PR adds the region-growing algorithm to DSWx-SAR.
The region-growing algorithm takes the fuzzy-value Geotiff and produces the intermediate binary water map, region_growing_output_binary_VV_VH.tif.

oberonia78 · 2023-07-13T18:17:21Z

src/dswx_sar/dswx_s1.py

+    # preprocessing (relocating ancillary data and filtering)
+    pre_processing.run(cfg)
+
+    # create dummy water map.


Suggested change

# create dummy water map.

# create dummy water map. This will be replaced or removed in future.

oberonia78 · 2023-07-13T18:34:47Z

src/dswx_sar/region_growing.py

+                data_shape,
+                pad_shape)
+        # run region-growing for blocks in parallel
+        result = Parallel(n_jobs=10)(delayed(process_region_growing_block)(


need to be fixed

Suggested change

result = Parallel(n_jobs=10)(delayed(process_region_growing_block)(

result = Parallel(n_jobs=-1)(delayed(process_region_growing_block)(

oberonia78 · 2023-07-13T18:39:38Z

src/dswx_sar/region_growing.py

+        water_binary[buffer_binary] = new_water
+        number_added = np.sum(new_water)
+        itercount = itercount + 1
+


print number of iteration and number of pixels added.

src/dswx_sar/dswx_sar_util.py

src/dswx_sar/region_growing.py

oberonia78 · 2023-08-17T10:21:17Z

@LiangJYu Thank you for the comments. I revised the PR based on your comments.

src/dswx_sar/region_growing.py

LiangJYu · 2023-08-21T01:45:21Z

src/dswx_sar/region_growing.py

+    Parameters
+    ----------
+    fuzz_image : numpy.ndarray
+        fuzzy image with values [0, 1] where 0 is non-water and 1 is water


Suggested change

fuzzy image with values [0, 1] where 0 is non-water and 1 is water

fuzzy image with values [0, 1] representing likelihood of water where 0 is 0% and 1 is 100%

Does the above better describe fuzz_image? If so, would water_likeihood_image be a more specific name?

Good suggestions. But, region-growing is used in different ways in initial thresholding, regardless of water. So more general name seems better.

How do you like likelihood_image? It's more specific than fuzz but still general since water is not in the variable name. Or is my interpretation of values as [0-1] as a likelihood incorrect or too narrow?

Also for clarification about the comment above, do you mean this region-growing code can/will be used for features besides water? If so, would it make sense remove water from variables names?

Sorry for all these questions...

src/dswx_sar/region_growing.py

LiangJYu · 2023-08-21T01:51:10Z

src/dswx_sar/region_growing.py

+    initial_seed : float
+        Threshold used for the seed of water body [0 ~ 1]


Suggested change

initial_seed : float

Threshold used for the seed of water body [0 ~ 1]

initial_water_threshold : float

Initial threshold [0 - 1] used to classify

`fuzz_image` into water and non-water pixels.

If a pixel values exceeds `initial_water_threshold`

then it is classified as water, otherwise it is

classified as non-water.

If I'm understanding the purpose of algorithm, I think the description above may better describe the purpose of this parameter.

For my curiosity, why is the initial threshold higher than tolerance?

Your description is more specific and better for this specific purpose. But in my opinion, including 'water' in the variable name may be too specific.

The reason why the initial threshold is always higher than tolerance is that the region growing starts from the initial threshold (seed) and classify the fuzzy image with the initial threshold to initially find the high likelihood of water and remove the wrong classified pixels.

After that, in the next loop, the code looks for adjacent pixels within the relaxed values to find more water areas. This is because some pixels in water objects can have slightly lower fuzzy values due to the windy conditions.

Thank for the explanation.

What do you think of either seed_threshold or initial_threshold? Without looking at the docstring I think either suggestion is more specific and there's a naming consistency that implies from the looking at the parameters of the relation to relaxed_threshold.

src/dswx_sar/region_growing.py

LiangJYu · 2023-08-21T01:56:48Z

src/dswx_sar/region_growing.py

+import time
+import os
+import numpy as np
+from scipy import ndimage
+import cv2
+
+import time
+import logging
+import mimetypes
+from joblib import Parallel, delayed


Imports are not ordered according to pep8. I recommend running this through a linter after the PR gets its 2 approvals to resolve.

src/dswx_sar/region_growing.py

oberonia78 · 2023-08-21T23:02:41Z

@LiangJYu Thank you for your great comments and corrections. I just finished the modification. Would you be able to have another look?

LiangJYu · 2023-08-29T03:52:53Z

src/dswx_sar/region_growing.py

+    nb_components, *_ = cv2.connectedComponentsWithStats(
+        thresh,
+        connectivity=8)
+    nb_components = nb_components - 1


Suggested change

nb_components = nb_components - 1

nb_components -= 1

nit - concise

LiangJYu · 2023-08-29T04:02:35Z

src/dswx_sar/region_growing.py

+    Parameters
+    ----------
+    fuzz_image : numpy.ndarray
+        fuzzy image with values [0, 1] where 0 is non-water and 1 is water


How do you like likelihood_image? It's more specific than fuzz but still general since water is not in the variable name. Or is my interpretation of values as [0-1] as a likelihood incorrect or too narrow?

Also for clarification about the comment above, do you mean this region-growing code can/will be used for features besides water? If so, would it make sense remove water from variables names?

Sorry for all these questions...

LiangJYu

Mostly nits. Sorry for not getting to this sooner.

LiangJYu · 2023-08-29T04:09:03Z

src/dswx_sar/region_growing.py

+                                initial_seed=0.6,
+                                relaxed_threshold=0.45,
+                                maxiter=200):
+    """Process region growing using parallel


Suggested change

"""Process region growing using parallel

"""Perform region growing in parallel

nit - wording

oberonia78 · 2023-08-30T01:14:47Z

@LiangJYu Thank you for your suggestions. I just removed 'water' from the variable names. likelihood_image seems reasonable to me. If you don't have additional comments on this PR, may I merge it?

#13 (comment)

LiangJYu

Mostly nits with a couple questions. Should be good to go after this.

LiangJYu · 2023-08-30T02:01:57Z

src/dswx_sar/region_growing.py

+        fuzzy image with values [0, 1]representing
+        likelihood of water where 0 is 0% and 1 is 100%


Suggested change

fuzzy image with values [0, 1]representing

likelihood of water where 0 is 0% and 1 is 100%

fuzzy image with values [0, 1] representing

likelihood of feature (e.g. water) where 0 is 0% and 1 is 100%

LiangJYu · 2023-08-30T02:05:42Z

src/dswx_sar/region_growing.py

+    likelihood_image : numpy.ndarray
+        fuzzy image with values [0, 1]representing
+        likelihood of water where 0 is 0% and 1 is 100%
+    initial_seed : float


What do you think of either seed_threshold or initial_threshold to replace initial_seed? Without needing to look at the docstring, I think either suggestion is more specific/descriptive and there's a naming consistency that implies a relation to relaxed_threshold.

LiangJYu · 2023-08-30T02:07:19Z

src/dswx_sar/region_growing.py

+        `fuzz_image` into water and non-water pixels.
+        If a pixel values exceeds `initial_seed`
+        then it is classified as water, otherwise it is
+        classified as non-water.
+    relaxed_threshold : float
+        relaxed threshold to be used for transient area
+        between water and non-water. Any pixel value
+        greater than threshold classified as water,
+        otherwise it's classified as non-water.


Suggested change

`fuzz_image` into water and non-water pixels.

If a pixel values exceeds `initial_seed`

then it is classified as water, otherwise it is

classified as non-water.

relaxed_threshold : float

relaxed threshold to be used for transient area

between water and non-water. Any pixel value

greater than threshold classified as water,

otherwise it's classified as non-water.

`fuzz_image` into feature and non-feature pixels.

If a pixel values exceeds `initial_seed`

then it is classified as feature, otherwise it is

classified as non-feature.

relaxed_threshold : float

relaxed threshold to be used for transient area

between feature and non-feature. Any pixel value

greater than threshold classified as feature,

otherwise it's classified as non-feature.

More replacements of water with feature.

LiangJYu · 2023-08-30T02:07:45Z

src/dswx_sar/region_growing.py

+        `likelihood_image` into water and non-water pixels.
+        If a pixel values exceeds `initial_seed`
+        then it is classified as water, otherwise it is
+        classified as non-water.
+    relaxed_threshold : float
+        relaxed threshold to be used for transient area
+        between water and non-water. Any pixel value
+        greater than threshold classified as water,
+        otherwise it's classified as non-water.


Replace water with feature in docstring.

LiangJYu · 2023-08-30T02:09:46Z

src/dswx_sar/region_growing.py

+        `likelihood_image` into water and non-water pixels.
+        If a pixel values exceeds `initial_seed`
+        then it is classified as water, otherwise it is
+        classified as non-water.
+    relaxed_threshold: float
+        value where region-growing stops.
+        relaxed threshold to be used for transient area
+        between water and non-water. Any pixel value
+        greater than threshold classified as water,
+        otherwise it's classified as non-water.


More replacement of water with feature needed in this docstring.

LiangJYu · 2023-08-30T02:16:04Z

src/dswx_sar/dswx_sar_util.py

+    data_block: np.ndarray
+        Block read from raster with shape specified in block_param.
+    '''
+


Suggested change

nit - whitespace/extra row

LiangJYu · 2023-08-30T02:16:16Z

src/dswx_sar/dswx_sar_util.py

+    block_param: BlockParam
+        Object specifying where and how much to write to out_raster.
+    '''
+


Suggested change

nit -white space/extra row

LiangJYu · 2023-08-30T02:16:42Z

src/dswx_sar/region_growing.py

+        1: the pixels involved in region growing (i.e., water)
+        0: the pixels not involved in region growing (i.e., non-water)
+    """
+


Suggested change

nit - white space/extra row

LiangJYu · 2023-08-30T02:16:57Z

src/dswx_sar/region_growing.py

+    data_block: numpy.ndarraya
+        fuzzy values after region growing
+    """
+


Suggested change

white space/extra row

LiangJYu · 2023-08-30T02:17:07Z

src/dswx_sar/region_growing.py

+    maxiter: integer
+        maximum number of dilation
+    """
+


Suggested change

white space/extra row

region growing algorithm

1b7211d

oberonia78 requested review from hfattahi, seongsujeong and gshiroma June 24, 2023 02:49

oberonia78 commented Jul 13, 2023

View reviewed changes

correct blank lines and format

e7cfc09

LiangJYu reviewed Aug 16, 2023

View reviewed changes

add comments and change variable names

b2ec6e7

Merge branch 'main' into add_region_growing

169fc6f

LiangJYu reviewed Aug 21, 2023

View reviewed changes

revised docstrings

bba1299

LiangJYu reviewed Aug 29, 2023

View reviewed changes

change variable name and docstring

9fd5d67

LiangJYu reviewed Aug 30, 2023

View reviewed changes

Jungkyo Jung and others added 3 commits August 29, 2023 23:27

update docstrings

c3cb639

remove white space

cb606c8

Merge branch 'main' into add_region_growing

5d9bf5f

oberonia78 merged commit 594c06a into opera-adt:main Aug 30, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add region growing algorithm #13

add region growing algorithm #13

oberonia78 commented Jun 24, 2023 •

edited

Loading

oberonia78 Jul 13, 2023

oberonia78 Jul 13, 2023

oberonia78 Jul 13, 2023

oberonia78 commented Aug 17, 2023

LiangJYu Aug 21, 2023

oberonia78 Aug 21, 2023

LiangJYu Aug 29, 2023

LiangJYu Aug 21, 2023

oberonia78 Aug 21, 2023

LiangJYu Aug 29, 2023 •

edited

Loading

LiangJYu Aug 21, 2023

oberonia78 commented Aug 21, 2023

LiangJYu Aug 29, 2023

LiangJYu Aug 29, 2023

LiangJYu left a comment

LiangJYu Aug 29, 2023

oberonia78 commented Aug 30, 2023

LiangJYu left a comment •

edited

Loading

LiangJYu Aug 30, 2023

LiangJYu Aug 30, 2023

LiangJYu Aug 30, 2023

LiangJYu Aug 30, 2023

LiangJYu Aug 30, 2023

LiangJYu Aug 30, 2023

LiangJYu Aug 30, 2023

LiangJYu Aug 30, 2023

LiangJYu Aug 30, 2023

LiangJYu Aug 30, 2023

	# create dummy water map.
	# create dummy water map. This will be replaced or removed in future.

	result = Parallel(n_jobs=10)(delayed(process_region_growing_block)(
	result = Parallel(n_jobs=-1)(delayed(process_region_growing_block)(

	fuzzy image with values [0, 1] where 0 is non-water and 1 is water
	fuzzy image with values [0, 1] representing likelihood of water where 0 is 0% and 1 is 100%

		initial_seed : float
		Threshold used for the seed of water body [0 ~ 1]

	"""Process region growing using parallel
	"""Perform region growing in parallel

		fuzzy image with values [0, 1]representing
		likelihood of water where 0 is 0% and 1 is 100%

add region growing algorithm #13

add region growing algorithm #13

Conversation

oberonia78 commented Jun 24, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

oberonia78 commented Aug 17, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

LiangJYu Aug 29, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

oberonia78 commented Aug 21, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

LiangJYu left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

oberonia78 commented Aug 30, 2023

LiangJYu left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

oberonia78 commented Jun 24, 2023 •

edited

Loading

LiangJYu Aug 29, 2023 •

edited

Loading

LiangJYu left a comment •

edited

Loading