Losing last mat row and column during `remap()` with `BORDER_TRANSPARENT` #23562

momasthagnum · 2023-04-28T13:23:35Z

System Information

OpenCV python version: 4.7.0
Operating System / Platform: Windows 11
Python revision: 3.10.8

Detailed description

[EDIT: unquoted relevant part of my question..]

Hi,

first off thanks for the great work you guys are doing!
I'll try and break this down with the attached python snippet.

The docs say

When borderMode=BORDER_TRANSPARENT, it means that the pixels in the destination image that corresponds to the "outliers" in the source image are not modified by the function.

and sadly I cannot seem to find much more on this anywhere, so I may be using it wrong. In that case apologies for creating this issue. But my understanding is that in the example code, this should lead to pixels in the destination staying black when trying to map to a pixel in the source that doesn't exist (e.g. 20). However, in the example, all pixels in the mapping mats exist in the source, but still, the last row and column in the destination always remain 0 when using BORDER_TRANSPARENT. It works as expected with BORDER_CONSTANT:

BORDER_CONSTANT

BORDER_TRANSPARENT

Steps to reproduce

bgImage = np.zeros((8, 12, 1), dtype=np.float32)

source = np.array([[0.1, 0.2, 0.3],
                   [0.5, 0.6, 0.7],
                   [0.9, 1.0, 0.9]], dtype=np.float32).reshape(3, 3, 1)

imageSize = bgImage.shape[1], bgImage.shape[0]

map1 = np.array([[0, 1, 2],
                 [0, 1, 2],
                 [0, 1, 2]], dtype=np.float32)

map2 = np.array([[0, 0, 0],
                 [1, 1, 1],
                 [2, 2, 2]], dtype=np.float32)

destination = np.zeros((3, 3, 1), dtype=np.float32)

cv2.remap(source,
          map1,
          map2,
          interpolation=cv2.INTER_LINEAR,
          dst=destination,
          borderMode=cv2.BORDER_TRANSPARENT)

result = bgImage.copy()
result[2:2+3, 2:2+3] = destination
upscaledImage = cv2.resize(result, np.array([1200, 800]), interpolation=cv2.INTER_NEAREST)

cv2.imshow("test", upscaledImage)
cv2.waitKey(0)
cv2.destroyAllWindows()

Issue submission checklist

I report the issue, it's not a question
I checked the problem with documentation, FAQ, open issues, forum.opencv.org, Stack Overflow, etc and have not found any solution
I updated to the latest OpenCV version and the issue is still there
There is reproducer code and related data files (videos, images, onnx, etc)

The text was updated successfully, but these errors were encountered:

dkurt · 2023-05-15T06:37:14Z

the pixels in the destination image

So try something like that:

destination = source.copy()

cv2.remap(source,
          map1,
          map2,
          interpolation=cv2.INTER_LINEAR,
          dst=destination,
          borderMode=cv2.BORDER_TRANSPARENT)

print(destination.reshape(3, 3))

[[0.1 0.2 0.3]
 [0.5 0.6 0.7]
 [0.9 1.  0.9]]

momasthagnum · 2023-05-15T07:37:23Z

the pixels in the destination image

So try something like that:

destination = source.copy()

cv2.remap(source,
          map1,
          map2,
          interpolation=cv2.INTER_LINEAR,
          dst=destination,
          borderMode=cv2.BORDER_TRANSPARENT)

print(destination.reshape(3, 3))

[[0.1 0.2 0.3]
 [0.5 0.6 0.7]
 [0.9 1.  0.9]]

Thanks for the reply. However, I'm not sure what to make of your instruction. If I set destination as a copy of source, the effect I'm describing won't be noticeable, since the pixel values are already set to the expected numbers. Do I get this part wrong?

my understanding is that in the example code, this should lead to pixels in the destination staying black when trying to map to a pixel in the source that doesn't exist (e.g. 20). However, in the example, all pixels in the mapping mats exist in the source, but still, the last row and column in the destination always remain 0 when using BORDER_TRANSPARENT

dkurt · 2023-06-07T11:53:19Z

@momasthagnum, please take a look at this proposal: #23754

momasthagnum · 2023-06-07T12:54:59Z

@dkurt, wow great! thanks so much for your effort! 😀

Keep inliers for linear remap with BORDER_TRANSPARENT #23754 Address #23562 ### Pull Request Readiness Checklist resolves #23562 I do think that this is a bug because with `INTER_CUBIC + BORDER_TRANSPARENT` the last column and row are preserved. So same should be done for `INTER_LINEAR` See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request - [x] I agree to contribute to the project under Apache 2 License. - [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV - [x] The PR is proposed to the proper branch - [x] There is a reference to the original bug report and related work - [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable Patch to opencv_extra has the same branch name. - [x] The feature is well documented and sample code can be built with the project CMake

I believe this is a proper fix to opencv#23562 The PR opencv#23754 overwrites data while that should not be the case with transparent data. The original test is failing because points at the border do not get computed because they do not have 4 neighbors to be computed. Still ,we can approximate their computation with whatever neighbors that are available.

Fix imgwarp at borders when transparent. #23922 I believe this is a proper fix to #23562 The PR #23754 overwrites data while that should not be the case with transparent data. The original test is failing because points at the border do not get computed because they do not have 4 neighbors to be computed. Still ,we can approximate their computation with whatever neighbors that are available. ### Pull Request Readiness Checklist See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request - [x] I agree to contribute to the project under Apache 2 License. - [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV - [x] The PR is proposed to the proper branch - [x] There is a reference to the original bug report and related work - [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable Patch to opencv_extra has the same branch name. - [x] The feature is well documented and sample code can be built with the project CMake

Keep inliers for linear remap with BORDER_TRANSPARENT opencv#23754 Address opencv#23562 ### Pull Request Readiness Checklist resolves opencv#23562 I do think that this is a bug because with `INTER_CUBIC + BORDER_TRANSPARENT` the last column and row are preserved. So same should be done for `INTER_LINEAR` See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request - [x] I agree to contribute to the project under Apache 2 License. - [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV - [x] The PR is proposed to the proper branch - [x] There is a reference to the original bug report and related work - [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable Patch to opencv_extra has the same branch name. - [x] The feature is well documented and sample code can be built with the project CMake

Fix imgwarp at borders when transparent. opencv#23922 I believe this is a proper fix to opencv#23562 The PR opencv#23754 overwrites data while that should not be the case with transparent data. The original test is failing because points at the border do not get computed because they do not have 4 neighbors to be computed. Still ,we can approximate their computation with whatever neighbors that are available. ### Pull Request Readiness Checklist See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request - [x] I agree to contribute to the project under Apache 2 License. - [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV - [x] The PR is proposed to the proper branch - [x] There is a reference to the original bug report and related work - [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable Patch to opencv_extra has the same branch name. - [x] The feature is well documented and sample code can be built with the project CMake

Keep inliers for linear remap with BORDER_TRANSPARENT opencv#23754 Address opencv#23562 ### Pull Request Readiness Checklist resolves opencv#23562 I do think that this is a bug because with `INTER_CUBIC + BORDER_TRANSPARENT` the last column and row are preserved. So same should be done for `INTER_LINEAR` See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request - [x] I agree to contribute to the project under Apache 2 License. - [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV - [x] The PR is proposed to the proper branch - [x] There is a reference to the original bug report and related work - [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable Patch to opencv_extra has the same branch name. - [x] The feature is well documented and sample code can be built with the project CMake

Fix imgwarp at borders when transparent. opencv#23922 I believe this is a proper fix to opencv#23562 The PR opencv#23754 overwrites data while that should not be the case with transparent data. The original test is failing because points at the border do not get computed because they do not have 4 neighbors to be computed. Still ,we can approximate their computation with whatever neighbors that are available. ### Pull Request Readiness Checklist See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request - [x] I agree to contribute to the project under Apache 2 License. - [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV - [x] The PR is proposed to the proper branch - [x] There is a reference to the original bug report and related work - [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable Patch to opencv_extra has the same branch name. - [x] The feature is well documented and sample code can be built with the project CMake

momasthagnum added the bug label Apr 28, 2023

dkurt mentioned this issue Jun 6, 2023

Keep inliers for linear remap with BORDER_TRANSPARENT #23754

Merged

6 tasks

asmorkalov closed this as completed in #23754 Jun 16, 2023

vrabaud mentioned this issue Jul 4, 2023

Fix imgwarp at borders when transparent. #23922

Merged

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Losing last mat row and column during `remap()` with `BORDER_TRANSPARENT` #23562

Losing last mat row and column during `remap()` with `BORDER_TRANSPARENT` #23562

momasthagnum commented Apr 28, 2023 •

edited

Loading

dkurt commented May 15, 2023

momasthagnum commented May 15, 2023

dkurt commented Jun 7, 2023

momasthagnum commented Jun 7, 2023

Losing last mat row and column during remap() with BORDER_TRANSPARENT #23562

Losing last mat row and column during remap() with BORDER_TRANSPARENT #23562

Comments

momasthagnum commented Apr 28, 2023 • edited Loading

System Information

Detailed description

Steps to reproduce

Issue submission checklist

dkurt commented May 15, 2023

momasthagnum commented May 15, 2023

dkurt commented Jun 7, 2023

momasthagnum commented Jun 7, 2023

Losing last mat row and column during `remap()` with `BORDER_TRANSPARENT` #23562

Losing last mat row and column during `remap()` with `BORDER_TRANSPARENT` #23562

momasthagnum commented Apr 28, 2023 •

edited

Loading