change binarization in findChessboardCorners() #24564

thewoz · 2023-11-20T16:04:54Z

In testing to solve problem #23558 I noticed that on certain images CALIB_CB_FAST_CHECK fails.
The problem arises in the binarization of the pictures which is done by a "classic" method.
This causes images like the following to be binarized badly and the check fails.

What I did was to eliminate icvBinarizationHistogramBased() and instead use adaptiveThreshold()

After the change I tried the function on other nine different checkerboard images and had no problems.

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

I agree to contribute to the project under Apache 2 License.
To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
The PR is proposed to the proper branch
There is a reference to the original bug report and related work
There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
The feature is well documented and sample code can be built with the project CMake

s-trinh · 2023-11-20T19:49:50Z

Maybe:

run the new method adaptiveThreshold()
if later the corners detection fails, fall back to the old approach?
or add a parameter to choose/customize the binarization method to use?

thewoz · 2023-11-20T20:16:26Z

perhaps it might make sense to tie it to the CALIB_CB_ADAPTIVE_THRESH parameter.
if it is defined the adaptive threshold is used if not the old method is used.
what do you think?

s-trinh · 2023-11-20T20:39:24Z

Yes, using a flag looks good to me.

thewoz · 2023-11-20T22:07:29Z

Hi,
I have test the pull request against the images:
BoardStereoL1-6.png BoardStereoR1-6.png and chess1-6.png

in

https://github.com/opencv/opencv_extra/tree/4.x/testdata/cv/cameracalibration

and the chessboard was found in each of them.

AleksandrPanov · 2023-11-21T09:04:05Z

[ RUN      ] Calib3d_StereoCalibrate_CPP.regression
/home/ci/opencv/modules/ts/src/ts.cpp:618: Failure
Failed

	failure reason: Bad accuracy
	test case #-1
	seed: 00000000000c5a5d
-----------------------------------
	LOG:
The distance between y coordinates is too big(=4.24615) (used uncalibrated stereo), testcase 1

@thewoz, you have regressions in accuracy.

thewoz · 2023-12-01T11:09:07Z

Hi I have been trying to figure out what is wrong.
What happens is that there are only 2 corners in one image, out of 28, that have an error of 4.24615px after rectifying the image on the y when the maximum is set to 4px. This causes the test to fail.

Investigating you more what comes out is that the problem comes from how findChessboardCorners() found the corners.
Since the binarization is different corners found come out slightly different.
findChessboardCorners() inside it calls cornerSubPix() with a very small mask (2x2) which in fact in practice is like not calling it.

In fact by calling cornerSubPix() externally from findChessboardCorners(), with a mask slightly larger than 6 pixels, as also suggested by the documentation of findChessboardCorners(), the problem is solved.

thewoz · 2023-12-14T08:36:52Z

hello sorry to bother you all the time.
is there anything i can do?

thewoz · 2023-12-14T15:45:48Z

now should work!

thewoz · 2024-01-04T10:08:31Z

Hi @asmorkalov, I should have resolved the conflict.

thewoz · 2024-05-09T09:51:49Z

hi,
is there anything I can do?

AleksandrPanov

Tested with benchmark --configuration=generate_run --board_x=7 --board_y=6 --path=res_chessboard --synthetic_object=chessboard:

Before PR:

After PR:

There are more improvements than regressions, but additional testing needs to be done.

AleksandrPanov · 2024-05-29T07:56:09Z

modules/calib3d/src/calibinit.cpp

+    if(!is_plain) {
+      if(flags & CALIB_CB_ADAPTIVE_THRESH)
+      {
+        // Assume that the checkerboard occupies 5% of the image
+        int min_size = cvRound((img.cols * img.rows * 0.05) / ((pattern_size.width+1) * (pattern_size.height+1)));
+        if(min_size%2==0) min_size += 1;
+        adaptiveThreshold(img, thresh_img_new, 255, ADAPTIVE_THRESH_MEAN_C, THRESH_BINARY, min_size, 0);
+      }
+      else
+      {
        icvBinarizationHistogramBased(thresh_img_new); // process image in-place
+      }
+    }


fix the tabs

AleksandrPanov · 2024-05-29T07:56:49Z

modules/calib3d/src/calibinit.cpp

-        goto finalize;
+      goto finalize;


does not apply to this PR

AleksandrPanov · 2024-05-29T07:58:48Z

modules/calib3d/test/test_cameracalibration.cpp

+
+            Mat leftGrey = left;
+            if(left.channels() != 1) cv::cvtColor(left, leftGrey, cv::COLOR_BGR2GRAY);
+
+            Mat rightGrey = right;
+            if(right.channels() != 1) cv::cvtColor(right, rightGrey, cv::COLOR_BGR2GRAY);
+
+            cv::cornerSubPix(leftGrey,  imgpt1[i], cv::Size(6,6), cv::Size(-1,-1), cv::TermCriteria(cv::TermCriteria::EPS + cv::TermCriteria::COUNT, 15, 0.1));
+            cv::cornerSubPix(rightGrey, imgpt2[i], cv::Size(6,6), cv::Size(-1,-1), cv::TermCriteria(cv::TermCriteria::EPS + cv::TermCriteria::COUNT, 15, 0.1));
+


Delete these changes and make a rebase from 4.x (cornerSubPix has already been added to 4.x)

AleksandrPanov · 2024-05-29T10:17:57Z

@thewoz, rebase needs to be fixed, you have 622 commits now)
Will you do it or do you need help?

thewoz · 2024-05-29T10:29:15Z

hi @AleksandrPanov,
maybe I'm doing the procedure wrong.
i did the rebase with the default branch 4.x
should i do it with another one? with upstream 4.x?

AleksandrPanov · 2024-05-29T11:10:32Z

@thewoz, update your local repo to main opencv repo and make rebase

git checkout 4.x
git pull https://github.com/opencv/opencv
git push https://github.com/thewoz/opencv
git checkout adaptive-findChessboardCorners
git rebase 4.x

thewoz · 2024-05-29T11:19:28Z

hi @AleksandrPanov
Sorry I don't want to make a mistake.
it asks me this.
how should i configure it?

git pull https://github.com/opencv/opencv
From https://github.com/opencv/opencv
 * branch                  HEAD       -> FETCH_HEAD
hint: You have divergent branches and need to specify how to reconcile them.
hint: You can do so by running one of the following commands sometime before
hint: your next pull:
hint: 
hint:   git config pull.rebase false  # merge
hint:   git config pull.rebase true   # rebase
hint:   git config pull.ff only       # fast-forward only
hint: 
hint: You can replace "git config" with "git config --global" to set a default
hint: preference for all repositories. You can also pass --rebase, --no-rebase,
hint: or --ff-only on the command line to override the configured default per
hint: invocation.
fatal: Need to specify how to reconcile divergent branches.

AleksandrPanov · 2024-05-29T11:37:12Z

@thewoz

hint: You have divergent branches and need to specify how to reconcile them.
hint: You can do so by running one of the following commands sometime before

Did you make your commits in your local 4.x?
Roll back your 4.x branch to the default state.

It is better to make changes to your branch and not to "main/master"

thewoz · 2024-05-30T11:25:46Z

Hi @AleksandrPanov,
I hope I was able to do everything you asked me to do.
I try to repeat the tests I had done last time?
The banchmarks I never figured out how to run them.

thewoz changed the title ~~Update calibinit.cpp~~ change binarization in findChessboardCorners() Nov 20, 2023

asmorkalov added the category: calib3d label Nov 20, 2023

asmorkalov requested review from asmorkalov, vpisarev and AleksandrPanov November 20, 2023 19:38

asmorkalov self-assigned this Nov 21, 2023

asmorkalov added this to the 4.9.0 milestone Nov 21, 2023

asmorkalov modified the milestones: 4.9.0, 4.10.0 Dec 20, 2023

AleksandrPanov modified the milestones: 4.10.0, 4.11.0 May 29, 2024

AleksandrPanov requested changes May 29, 2024

View reviewed changes

thewoz force-pushed the adaptive-findChessboardCorners branch from c7d8e1d to 314208f Compare May 29, 2024 09:17

thewoz force-pushed the adaptive-findChessboardCorners branch from 549ec68 to 774efd1 Compare May 30, 2024 09:20

use adaptiveThreshold instead icvBinarizationHistogramBased

95d21c7

AleksandrPanov force-pushed the adaptive-findChessboardCorners branch from ec45804 to 95d21c7 Compare June 16, 2024 21:21

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

change binarization in findChessboardCorners() #24564

change binarization in findChessboardCorners() #24564

thewoz commented Nov 20, 2023

s-trinh commented Nov 20, 2023

thewoz commented Nov 20, 2023

s-trinh commented Nov 20, 2023

thewoz commented Nov 20, 2023

AleksandrPanov commented Nov 21, 2023

thewoz commented Dec 1, 2023

thewoz commented Dec 14, 2023

thewoz commented Dec 14, 2023

thewoz commented Jan 4, 2024 •

edited

thewoz commented May 9, 2024

AleksandrPanov left a comment

AleksandrPanov May 29, 2024

thewoz May 29, 2024

AleksandrPanov May 29, 2024

thewoz May 29, 2024

AleksandrPanov May 29, 2024

thewoz May 29, 2024

AleksandrPanov commented May 29, 2024

thewoz commented May 29, 2024 •

edited

AleksandrPanov commented May 29, 2024 •

edited

thewoz commented May 29, 2024

AleksandrPanov commented May 29, 2024

thewoz commented May 30, 2024

change binarization in findChessboardCorners() #24564

Are you sure you want to change the base?

change binarization in findChessboardCorners() #24564

Conversation

thewoz commented Nov 20, 2023

Pull Request Readiness Checklist

s-trinh commented Nov 20, 2023

thewoz commented Nov 20, 2023

s-trinh commented Nov 20, 2023

thewoz commented Nov 20, 2023

AleksandrPanov commented Nov 21, 2023

thewoz commented Dec 1, 2023

thewoz commented Dec 14, 2023

thewoz commented Dec 14, 2023

thewoz commented Jan 4, 2024 • edited

thewoz commented May 9, 2024

AleksandrPanov left a comment

Choose a reason for hiding this comment

AleksandrPanov May 29, 2024

Choose a reason for hiding this comment

thewoz May 29, 2024

Choose a reason for hiding this comment

AleksandrPanov May 29, 2024

Choose a reason for hiding this comment

thewoz May 29, 2024

Choose a reason for hiding this comment

AleksandrPanov May 29, 2024

Choose a reason for hiding this comment

thewoz May 29, 2024

Choose a reason for hiding this comment

AleksandrPanov commented May 29, 2024

thewoz commented May 29, 2024 • edited

AleksandrPanov commented May 29, 2024 • edited

thewoz commented May 29, 2024

AleksandrPanov commented May 29, 2024

thewoz commented May 30, 2024

thewoz commented Jan 4, 2024 •

edited

thewoz commented May 29, 2024 •

edited

AleksandrPanov commented May 29, 2024 •

edited