Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge 4.x -> 5.x #25745

Merged
merged 184 commits into from
Jun 27, 2024
Merged

Merge 4.x -> 5.x #25745

merged 184 commits into from
Jun 27, 2024

Conversation

mshabunin
Copy link
Contributor

@mshabunin mshabunin commented Jun 11, 2024

My attempt to make regular merge 🙂

Companion PRs: opencv/opencv_contrib#3758 and opencv/opencv_extra#1183

PR list:
a03b813 Merge pull request #25732 from dkurt:opencv_js_tests_update
8e93070 Merge pull request #25738 from rlexmann:imgproc_fix_divSpectrums
2629688 Merge pull request #25706 from cudawarped:fix_cuda_first_python_dep
3ea7ed9 Merge pull request #25718 from savuor:rv/hwasan_flag
bdf986e Merge pull request #25726 from chacha21:remap_relative_doc
cc6f85e Merge pull request #25427 from MaximSmolskiy:make-finding-corner-neighbor-symmetrical-in-ChessBoardDetector-findQuadNeighbors
3282954 Merge pull request #25723 from mshabunin:fix-ts-rng
3700f9e Merge pull request #25709 from dkurt:wrap_addLayer
bef5a87 Merge pull request #25722 from AleksandrPanov:update_testSeveralBoardsWithCustomIds
adcb070 Merge pull request #25307 from MaximMilashchenko:halrvv071
cbf3b11 Merge pull request #25720 from VadimLevin:dev/vlevin/floodFill-optional-mask
0d1ed49 Merge pull request #25703 from mshabunin:fix-imread-arg
a5d9c29 Merge pull request #25716 from alexlyulkov:al/mediandk-fix
92b588f Merge pull request #25702 from asmorkalov:as/gapi_disable_steaming_again
d8f0838 Merge pull request #25701 from keanep:I25700
a7e53aa Merge pull request #25671 from savuor:rv/arithm_extend_tests
1bd5ca1 Merge pull request #25686 from Kumataro:fix25674
1db6a8a Merge pull request #25665 from vrabaud:jacobian
98b8825 Merge pull request #25613 from CNOCycle:tflite/ops
29f91a0 Merge pull request #25680 from mshabunin:fix-approx-2
0810795 Merge pull request #25682 from fengyuentau:calib3d/lapack_fix_calib3d
472eba3 Merge pull request #25673 from AleksandrPanov:fix_charuco_board_markers
dcce2b8 Merge pull request #25662 from ujjayant-kadian:uk/port-gapi-onnxrt-backend-into-v2-api
d7f04a9 Merge pull request #25660 from Abdurrahheem:ash/fix-slice-empty-input
f4ebf7c Merge pull request #25679 from asmorkalov:as/HAL_min_max_idx_fix
7e9ef4d Merge pull request #25625 from fengyuentau:core/deploy_fix_lapack_ilp64
3e3ee10 Merge pull request #25658 from asmorkalov:as/variadic_tuple
2624929 Merge pull request #25672 from mshabunin:fix-approx
79af357 Merge pull request #25668 from vrabaud:legacy
d9421ac Merge pull request #25167 from plctlab:rvp_3rdparty
05e4860 Merge pull request #25412 from ZelboK:update-cudnn-to-9
b659351 Merge pull request #25647 from Kumataro:fix25646
c5976f7 Merge pull request #25641 from vrabaud:lsh
f1890e3 Merge pull request #25650 from sturkmen72:libjpeg-turbo
1fa96b1 Merge pull request #25616 from savuor:rv/yuv_docs
d97df26 Merge pull request #25623 from asmorkalov:as/jpegturbo_3.0.3
7b31cc7 Merge pull request #24897 from JStech:fix-handeye
cc49aee Merge pull request #25634 from savuor:rv/boxfilter_tests
b267f17 Merge pull request #25633 from savuor:rv/rotate_tests
f17b122 Merge pull request #25638 from bebuch:patch-2
ba28b7d Merge pull request #25636 from FantasqueX:use-left-arrow-1
28c3000 Merge pull request #25629 from arobase-che:fix-opencv.js-compilation-memory-init-file
f1aef89 Merge pull request #25619 from opencv-pushbot:gitee/alalek/update_ffmpeg_4.x
f5793d2 Merge pull request #25632 from bebuch:patch-1
de3eb0e Merge pull request #25626 from asmorkalov:as/HAL_lut_size
4824354 Merge pull request #25631 from asmorkalov:as/png_build_fix
16b5096 Merge pull request #25618 from asmorkalov:as/kleidicv_0.1.0
9f15595 Merge pull request #25627 from vrabaud:tiff
f2c48e5 Merge pull request #25622 from lackhole:remove-unicode-character
e765c9f Merge pull request #25580 from sturkmen72:libpng_1_6_43
8393885 Merge pull request #25615 from asmorkalov:update_version_4.10.0-pre
49f80cb Merge pull request #24804 from fengyuentau:fix_lapack_warnings
e525401 Merge pull request #25586 from hanliutong:rvv-64f
c71d495 Merge pull request #25611 from asmorkalov:as/openjpeg_2.5.2
23ba63b Merge pull request #25574 from savuor:rv/hal_sub8x32f
69af621 Merge pull request #25506 from savuor:rv/hal_mul16
83e32c4 Merge pull request #25511 from savuor:rv/hal_projectpoints
e05ad56 Merge pull request #24903 from cristidbr-adapta:feature-barcode-detector-parameters
6350bfb Merge pull request #25564 from mshabunin:cleanup-imgproc-2
8aa129d Merge pull request #25603 from asmorkalov:as/skip_vulkan_concat
5e78878 Merge pull request #25525 from sturkmen72:png-spng-test
bc0618b Merge pull request #25582 from fengyuentau:dnn/dump_pbtxt
0044047 Merge pull request #25598 from asmorkalov:as/tables_range_check_core
41f0898 Merge pull request #25585 from FantasqueX:add-policy-0148
f2d6527 Merge pull request #25596 from asmorkalov:as/ci_ubuntu24.04
d29ad2f Merge pull request #25443 from asmorkalov:as/kleidicv_hal
78ed6de Merge pull request #25594 from LaurentBerger:I25587
7713c84 Merge pull request #25297 from CNOCycle:tflite/transpose
76d9f7a Merge pull request #25591 from LaurentBerger:I25589
68581e7 Merge pull request #25592 from asmorkalov:as/gapi_disable_steaming_again
4d4def2 Merge pull request #25590 from asmorkalov:as/restore_highgui_test
47a6ffb Merge pull request #25561 from Kumataro:fix25560
03507e0 Merge pull request #25518 from alexlyulkov:al/fixed-gemm-openvino
f4e5438 Merge pull request #25566 from asmorkalov:as/moments_documentation
02ad8c2 Merge pull request #25577 from FantasqueX:fix-zlib-ng-version-parse-1
58e007e Merge pull request #25583 from mshabunin:fix-emd-32bit
3c7e94b Merge pull request #25576 from mshabunin:fix-env-doc
8dc9ff5 Merge pull request #25551 from Kumataro:fix25550
1d9a412 Merge pull request #25565 from savuor:rv/hal_eq_hist
1f1ba7e Merge pull request #25563 from asmorkalov:as/HAL_min_max_idx
5bd64e0 Merge pull request #25554 from savuor:rv/hal_lut
faa259a Merge pull request #25553 from asmorkalov:as/HAL_min_max_idx
a9e489f Merge pull request #25558 from asmorkalov:as/moments_sanity
ba65d2e Merge pull request #25549 from mshabunin:fix-cap-images-noise
8bc90a4 Merge pull request #25484 from mshabunin:bounding-rect-alignment
26a5730 Merge pull request #25542 from mshabunin:fix-disable-diagram
a4486c0 Merge pull request #25541 from mshabunin:fix-numpy2x-test
70d333d Merge pull request #25483 from asmorkalov:as/HAL_meanStdDev
c43957c Merge pull request #25538 from asmorkalov:as/resnet_optional
75f3f0f Merge pull request #25526 from NlSEMONO:patch-1
ac9a858 Merge pull request #25524 from alexlyulkov:al/openvino-layers
6e5a53c Merge pull request #25532 from asmorkalov:as/HAL_moments_bug
d9555db Merge pull request #25535 from asmorkalov:as/convertfp16_imgcodecs
c539cfd Merge pull request #25435 from WanliZhong:fcn2onnx
6bf758e Merge pull request #24782 from FantasqueX:4.x-zlib-ng
7ea4364 Merge pull request #25528 from asmorkalov:as/python_hdr_parser_fix
bc95f27 Merge pull request #25509 from savuor:rv/hal_otsu
ce642e9 Merge pull request #25510 from Kumataro:fix25497
1c9efbd Merge pull request #25523 from Kumataro:fix25522
b5cf4f1 Merge pull request #25514 from xndcn:patch-1
1f6f092 Merge pull request #25513 from zihaomu:fix_25498
cbf71c3 Merge pull request #25502 from Kumataro:fix25501
2a2ff55 Merge pull request #25496 from Kumataro:fix25495
b1e0197 Merge pull request #25308 from dkurt:not_normalized_findHomography
12e2cc9 Merge pull request #25491 from savuor:rv/hal_norm_hamming
357b9ab Merge pull request #25450 from savuor:rv/svd_perf
8af7669 Merge pull request #25493 from vrabaud:tiff
2cd3304 Merge pull request #25490 from asmorkalov:as/HAL_moments
7b58ffe Merge pull request #25486 from asmorkalov:as/gapi_warning_fix
2225b25 Merge pull request #25488 from inkredibl:doc-fix-findEssentialMat
ebea657 Merge pull request #24938 from DariaMityagina:icv/dm/add-media-frame-support-to-govbackend
7e56908 Merge pull request #25469 from mshabunin:cpp-emd
5b08437 Merge pull request #25453 from Kumataro:fix25452
616a913 Merge pull request #25468 from WanliZhong:change_js_ocvFaceDet_sample
50b6edc Merge pull request #25321 from dkurt:skip_gapi_test
ae85e51 Merge pull request #25423 from LuukvandenBent:CalibrateHandEyeDatatypeFix
3f8b56e Merge pull request #25459 from Kumataro:investigate_tiff_test

Previous "Merge 4.x": #25460

FantasqueX and others added 30 commits January 14, 2024 14:58
Zlib-ng is zlib replacement with optimizations for "next generation" systems. Its optimization may benifits image library decode and encode speed such as libpng. In our tests, if using zlib-ng and libpng combination on a x86_64 machine with AVX2, the time of `imdecode` amd `imencode` will drop 20% approximately. This patch enables zlib-ng's optimization if `CV_DISABLE_OPTIMIZATION` is OFF. Since Zlib-ng can dispatch intrinsics on the fly, port work is much easier.

Related discussion: opencv#22573
imgcodecs: tiff: test: ASSERT_NO_THROW() for imwrite/imread
…atatypeFix

Calibrate hand eye datatype fix opencv#25423

Fix for issue opencv#25421.

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Skip InferList.TestStreamingInfer
…sample

Replace opencv_face_detector with YuNet in js_face_recognition sample
imgcodecs: jpeg: refactoring for JpegEncoder
imgproc: refactor EMD to reduce C-API usage opencv#25469

- added more tests for EMD
- refactored to remove CvArr
- used BufferArea for memory allocations
- renamed functions and variables and formatted the code
- kept legacy functions intact in separate header
…frame-support-to-govbackend

G-API OV backend requires cv::MediaFrame opencv#24938

### Pull Request Readiness Checklist

**Background_subtraction demo G-API issue. Update:**

Porting to API20 resulted in an error (both for CPU and NPU):
```
[ERROR] OpenCV(4.9.0-dev) /home/runner/work/open_model_zoo/open_model_zoo/cache/opencv/modules/gapi/src/backends/ov/govbackend.cpp:813: error: (-215: assertion not done ) cv::util::holds_alternative<cv::GMatDesc>(input_meta) in function 'cfgPreProcessing'
```

Adding cv::MediaFrame support to govbackend resulted in the following (tested with CPU):
<img width="941" alt="image" src="https://github.com/opencv/opencv/assets/52502732/3a003d61-bda7-4b1e-9117-3410cda1ba32">

### TODO

- [ ] **As part of the review process [this comment](opencv#24938 (comment)) was addressed which make it impossible to run the demo. I will bring those changes back in a separate PR [support `PartialShape`]**

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
Fix documentation for findEssentialMat to reflect how it actually works. opencv#25488

Documentation for findEssentialMat() incorrectly states that the method uses the same cameraMatrix for both lists of points even though there are two cameraMatrix and distCoeffs.

Checked the code and it does the right thing i.e. uses cameraMatrix1, distCoeffs1 for points1 and cameraMatrix2, distCoeffs2 for points2.

Updated the documentation for the method to clarify what it does. The code itself is not changed.
That enables the numpy code to be its own library, in case
some users want to (e.g. CLIF library).
Replace static numpy allocator by function containing static.
Perf tests for SVD and solve() created opencv#25450

fixes opencv#25336

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
HAL for Hamming norm added opencv#25491

fixes opencv#25474

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Not-normalized output from findHomography opencv#25308

### Pull Request Readiness Checklist

resolves opencv#25133
resolves opencv#4834
resolves opencv#22166
resolves opencv#18592

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
highgui: wayland: show "NO" status if dependency is missing opencv#25496

Close opencv#25495

- [doc] Add document to enable Wayland highgui-backend in ubuntu 24.04.
- [build] Show "NO" status instead of version if dependency library is missing.
- [build] Fix to find Wayland EGL.
- [fix] Add some callback stub functions to suppress build warning.

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [ ] I agree to contribute to the project under Apache 2 License.
- [ ] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [ ] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
highgui: wayland: reduce cpu usage for cv::waitKey()
It is said "see above" because calcBackProject is listed under calcHist function in source file, while it is listed before due to the lexicographic ordering.
Fix doc about uniform parameter for imgproc calcBackProject function
savuor and others added 4 commits June 11, 2024 10:12
Android SDK build script: HWAsan support added opencv#25718

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
…_dep

`cuda`: Add missing python CUDA dependency when CUDA is a first class language
Fix arithmetics in CV_32F branch of divSpectrums()
Fix OpenCV.js tests opencv#25732

### Pull Request Readiness Checklist

* Firefox tests passed

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
@mshabunin
Copy link
Contributor Author

@dkurt , @vrabaud , please help with an issue with new 3d test "Calib3d_Homography.Refine"

[ RUN      ] Calib3d_Homography.Refine
/work/opencv/modules/3d/test/test_homography.cpp:763: Failure
Expected: (sqrt(error.at<double>(0, 0))) <= (method == LMEDS ? 7e-4 : 7e-5), actual: 0.000165377 vs 7e-05
/work/opencv/modules/3d/test/test_homography.cpp:763: Failure
Expected: (sqrt(error.at<double>(0, 0))) <= (method == LMEDS ? 7e-4 : 7e-5), actual: 0.000165377 vs 7e-05
/work/opencv/modules/3d/test/test_homography.cpp:763: Failure
Expected: (sqrt(error.at<double>(0, 0))) <= (method == LMEDS ? 7e-4 : 7e-5), actual: 0.0100815 vs 0.0007
[  FAILED  ] Calib3d_Homography.Refine (2 ms)

It is related to PRs #25665 and #25308. I'm not sure whether it is a merge issue or a test problem. Should we just change test threshold or fix something else?

@vrabaud
Copy link
Contributor

vrabaud commented Jun 12, 2024

@mshabunin , it seems you are not including all of #25665 Only missing: Jptr += 18; line 405 in 3d/src/fundam.cpp

@mshabunin
Copy link
Contributor Author

Yes, thank you! I added this change and test works better now. Though one case is still failing (LMEDS):

[ RUN      ] Calib3d_Homography.Refine
/work/opencv/modules/3d/test/test_homography.cpp:763: Failure
Expected: (sqrt(error.at<double>(0, 0))) <= (method == LMEDS ? 7e-4 : 7e-5), actual: 0.0041369 vs 0.0007
[  FAILED  ] Calib3d_Homography.Refine (2 ms)

@mshabunin
Copy link
Contributor Author

Maybe it is because of difference in Levenberg–Marquardt implementation between 4.x and 5.x? (implemented here #21018)

@vrabaud
Copy link
Contributor

vrabaud commented Jun 12, 2024

That test was added to exercise the Jacobian usage. Indeed, if the LM solver is different, that does not help. Maybe you can try increasing the number of iterations from 10 to 20 to see if that improves the error ?

@dkurt
Copy link
Member

dkurt commented Jun 12, 2024

@vrabaud, seems like it always 10:

.setMaxIterations(10)

@mshabunin, I think it’s ok to increase threshold for 5.x. As we see, with Jacobian fix the error decreased from 0.0100815 to 0.0041369. The test is quite synthetic and catches Jacobian patch only.

@mshabunin
Copy link
Contributor Author

Thank you! I tried to increase number of iterations, but it did not help. So I've updated the test threshold.

@mshabunin
Copy link
Contributor Author

@alexlyulkov , please take a look at the dnn part of this PR - whether I've correctly merged parts with OpenVINO support (there were discrepancies between #25524 and #25458).

@mshabunin mshabunin marked this pull request as ready for review June 14, 2024 10:46
@asmorkalov
Copy link
Contributor

asmorkalov commented Jun 25, 2024

minmax.dispatch.cpp changes are missing:

diff --git a/modules/core/src/minmax.dispatch.cpp b/modules/core/src/minmax.dispatch.cpp
index 6c50cb20b5..cd778b0283 100644
--- a/modules/core/src/minmax.dispatch.cpp
+++ b/modules/core/src/minmax.dispatch.cpp
@@ -309,6 +309,33 @@ void cv::minMaxIdx(InputArray _src, double* minVal,
                ocl_minMaxIdx(_src, minVal, maxVal, minIdx, maxIdx, _mask))
 
     Mat src = _src.getMat(), mask = _mask.getMat();
+
+    if (src.dims <= 2)
+    {
+        CALL_HAL(minMaxIdx, cv_hal_minMaxIdx, src.data, src.step, src.cols*cn, src.rows,
+                 src.depth(), minVal, maxVal, minIdx, maxIdx, mask.data);
+    }
+    else if (src.isContinuous())
+    {
+        int res = cv_hal_minMaxIdx(src.data, 0, (int)src.total()*cn, 1, src.depth(),
+                                   minVal, maxVal, minIdx, maxIdx, mask.data);
+
+        if (res == CV_HAL_ERROR_OK)
+        {
+            // minIdx[0] and minIdx[0] are always 0 for "flatten" version
+            if (minIdx)
+                ofs2idx(src, minIdx[1], minIdx);
+            if (maxIdx)
+                ofs2idx(src, maxIdx[1], maxIdx);
+            return;
+        }
+        else if (res != CV_HAL_ERROR_NOT_IMPLEMENTED)
+        {
+            CV_Error_(cv::Error::StsInternal,
+            ("HAL implementation minMaxIdx ==> " CVAUX_STR(cv_hal_minMaxIdx) " returned %d (0x%08x)", res, res));
+        }
+    }
+
     MinMaxIdxFunc func = getMinMaxIdxFunc(depth);
     CV_Assert( func != 0 );

modules/imgproc/test/test_contours_new.cpp Outdated Show resolved Hide resolved
modules/dnn/src/layers/scatter_layer.cpp Outdated Show resolved Hide resolved
modules/dnn/src/layers/scatterND_layer.cpp Outdated Show resolved Hide resolved
modules/dnn/src/layers/gather_layer.cpp Outdated Show resolved Hide resolved
modules/dnn/src/layers/gather_elements_layer.cpp Outdated Show resolved Hide resolved
int lda = (int)(a_step / sizeof(fptype)), sign = 0;
std::vector<int> piv(m+1);
#if defined (ACCELERATE_NEW_LAPACK) && defined (ACCELERATE_LAPACK_ILP64)
cv::AutoBuffer<long> piv_buff(m);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just for reference. m+1 change was discussed here: https://github.com/opencv/opencv/pull/18571/files#r1638087554

@param rt_data Pointer to 3x4 array containing rotation-then-translation matrix
@param intr_data Pointer to camera intrinsics structure
*/
inline int hal_ni_project_points_pinhole64f(const double* src_data, size_t src_step, size_t src_size,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The integration part of HAL is missing in the PR. undistortPoints was reworked in 5.x. I'll attach solution soon.

@mshabunin
Copy link
Contributor Author

I've copied that code block to minmax.dispatch.cpp. Perhaps we need to partially backport minmax changes to 4.x to avoid issues in future (is it #24962?).

@asmorkalov
Copy link
Contributor

asmorkalov commented Jun 26, 2024

@mshabunin The patch for the projectPoints HAL:

diff --git a/modules/3d/src/calibration_base.cpp b/modules/3d/src/calibration_base.cpp
index e087111efb..e3886375cd 100644
--- a/modules/3d/src/calibration_base.cpp
+++ b/modules/3d/src/calibration_base.cpp
@@ -41,6 +41,7 @@
 //M*/
 
 #include "precomp.hpp"
+#include "hal_replacement.hpp"
 #include "distortion_model.hpp"
 #include <stdio.h>
 #include <iterator>
@@ -539,20 +540,15 @@ void cv::projectPoints( InputArray _objectPoints,
               (objectPoints.rows == count && objpt_cn*objectPoints.cols == 3) ||
               (objectPoints.rows == 3 && objpt_cn == 1 && objectPoints.cols == count));
 
-    Mat matM(objectPoints.size(), CV_64FC(objpt_cn));
-    objectPoints.convertTo(matM, CV_64F);
     if (objectPoints.rows == 3 && objectPoints.cols == count) {
         Mat temp;
-        transpose(matM, temp);
-        matM = temp;
+        transpose(objectPoints, temp);
+        objectPoints = temp;
     }
 
     CV_Assert( _imagePoints.needed() );
     _imagePoints.create(count, 1, CV_MAKETYPE(objpt_depth, 2), -1, true);
     Mat ipoints = _imagePoints.getMat();
-    ipoints.convertTo(_m, CV_64F);
-    const Point3d* M = matM.ptr<Point3d>();
-    Point2d* m = _m.ptr<Point2d>();
 
     Mat rvec = _rvec.getMat(), tvec = _tvec.getMat();
     if(!((rvec.depth() == CV_32F || rvec.depth() == CV_64F) &&
@@ -656,6 +652,127 @@ void cv::projectPoints( InputArray _objectPoints,
     bool calc_derivatives = dpdr.data || dpdt.data || dpdf.data ||
                             dpdc.data || dpdk.data || dpdo.data;
 
+    if (!calc_derivatives)
+    {
+        if (objpt_depth == CV_32F && ipoints.type() == CV_32F)
+        {
+            float rtMatrix[12] = { (float)R[0], (float)R[1], (float)R[2], (float)t[0],
+                                (float)R[3], (float)R[4], (float)R[5], (float)t[1],
+                                (float)R[6], (float)R[7], (float)R[8], (float)t[2] };
+
+            cv_camera_intrinsics_pinhole_32f intr;
+            intr.fx = (float)fx; intr.fy = (float)fy;
+            intr.cx = (float)cx; intr.cy = (float)cy;
+            intr.amt_k = 0; intr.amt_p = 0; intr.amt_s = 0; intr.use_tau = false;
+
+            switch (ktotal)
+            {
+            case  0: break;
+            case  4: // [k_1, k_2, p_1, p_2]
+                intr.amt_k = 2; intr.amt_p = 2;
+                break;
+            case  5: // [k_1, k_2, p_1, p_2, k_3]
+                intr.amt_k = 3; intr.amt_p = 2;
+                break;
+            case  8: // [k_1, k_2, p_1, p_2, k_3, k_4, k_5, k_6]
+                intr.amt_k = 6; intr.amt_p = 2;
+                break;
+            case 12: // [k_1, k_2, p_1, p_2, k_3, k_4, k_5, k_6, s_1, s_2, s_3, s_4]
+                intr.amt_k = 6; intr.amt_p = 2; intr.amt_s = 4;
+                break;
+            case 14: // [k_1, k_2, p_1, p_2, k_3, k_4, k_5, k_6, s_1, s_2, s_3, s_4, tau_x, tau_y]
+                intr.amt_k = 6; intr.amt_p = 2; intr.amt_s = 4; intr.use_tau = true;
+                break;
+            default:
+                CV_Error(cv::Error::StsInternal, "Wrong number of distortion coefficients");
+            }
+
+            intr.k[0] = (float)k[0];
+            intr.k[1] = (float)k[1];
+            intr.k[2] = (float)k[4];
+            intr.k[3] = (float)k[5];
+            intr.k[4] = (float)k[6];
+            intr.k[5] = (float)k[7];
+
+            intr.p[0] = (float)k[2];
+            intr.p[1] = (float)k[3];
+
+            for (int ctr = 0; ctr < 4; ctr++)
+            {
+                intr.s[ctr] = (float)k[8+ctr];
+            }
+
+            intr.tau_x = (float)k[12];
+            intr.tau_y = (float)k[13];
+
+            CALL_HAL(projectPoints, cv_hal_project_points_pinhole32f,
+                     (float*)objectPoints.data, objectPoints.step, count,
+                     (float*)ipoints.data, ipoints.step, rtMatrix, &intr);
+        }
+
+        if (objpt_depth == CV_64F && ipoints.type() == CV_64F)
+        {
+            double rtMatrix[12] = { R[0], R[1], R[2], t[0],
+                                    R[3], R[4], R[5], t[1],
+                                    R[6], R[7], R[8], t[2] };
+
+            cv_camera_intrinsics_pinhole_64f intr;
+            intr.fx = fx; intr.fy = fy;
+            intr.cx = cx; intr.cy = cy;
+            intr.amt_k = 0; intr.amt_p = 0; intr.amt_s = 0; intr.use_tau = false;
+
+            switch (ktotal)
+            {
+            case  0: break;
+            case  4: // [k_1, k_2, p_1, p_2]
+                intr.amt_k = 2; intr.amt_p = 2;
+                break;
+            case  5: // [k_1, k_2, p_1, p_2, k_3]
+                intr.amt_k = 3; intr.amt_p = 2;
+                break;
+            case  8: // [k_1, k_2, p_1, p_2, k_3, k_4, k_5, k_6]
+                intr.amt_k = 6; intr.amt_p = 2;
+                break;
+            case 12: // [k_1, k_2, p_1, p_2, k_3, k_4, k_5, k_6, s_1, s_2, s_3, s_4]
+                intr.amt_k = 6; intr.amt_p = 2; intr.amt_s = 4;
+                break;
+            case 14: // [k_1, k_2, p_1, p_2, k_3, k_4, k_5, k_6, s_1, s_2, s_3, s_4, tau_x, tau_y]
+                intr.amt_k = 6; intr.amt_p = 2; intr.amt_s = 4; intr.use_tau = true;
+                break;
+            default:
+                CV_Error(cv::Error::StsInternal, "Wrong number of distortion coefficients");
+            }
+
+            intr.k[0] = k[0];
+            intr.k[1] = k[1];
+            intr.k[2] = k[4];
+            intr.k[3] = k[5];
+            intr.k[4] = k[6];
+            intr.k[5] = k[7];
+
+            intr.p[0] = k[2];
+            intr.p[1] = k[3];
+
+            for (int ctr = 0; ctr < 4; ctr++)
+            {
+                intr.s[ctr] = k[8+ctr];
+            }
+
+            intr.tau_x = k[12];
+            intr.tau_y = k[13];
+
+            CALL_HAL(projectPoints, cv_hal_project_points_pinhole64f,
+                     (double*)objectPoints.data, objectPoints.step, count,
+                     (double*)ipoints.data, ipoints.step, rtMatrix, &intr);
+        }
+    }
+
+    Mat matM(objectPoints.size(), CV_64FC(objpt_cn));
+    objectPoints.convertTo(matM, CV_64F);
+    ipoints.convertTo(_m, CV_64F);
+    const Point3d* M = matM.ptr<Point3d>();
+    Point2d* m = _m.ptr<Point2d>();
+
     for( i = 0; i < count; i++ )
     {
         double X = M[i].x, Y = M[i].y, Z = M[i].z;

@savuor please review the proposed change.

@mshabunin
Copy link
Contributor Author

@asmorkalov , I've added fix for projectPoints.

Copy link
Contributor

@asmorkalov asmorkalov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@asmorkalov
Copy link
Contributor

@mshabunin please squash the last 2 commits into merge commit to have clean git history after the merge.

@asmorkalov asmorkalov merged commit 26ea34c into opencv:5.x Jun 27, 2024
23 of 25 checks passed
@mshabunin mshabunin deleted the merge-4-5 branch June 27, 2024 18:22
@asmorkalov asmorkalov mentioned this pull request Jul 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet