2023

2023 OpenCV Technical Committee

Meeting_notes

Template

2023-mm-dd

Agenda

Minutes

To Dos

Name
- todo

2023-12-20

Vadim:
- [OpenCV China] PR with faster MatMul in DNN for faster transformers (https://github.com/opencv/opencv/pull/24694) has been merged. Attention acceleration PR (https://github.com/opencv/opencv/pull/24476) is almost ready; will try to squeeze it into 4.9.0.
- Regression in convolution is likely fixed (https://github.com/opencv/opencv/pull/24709). Depth-wise convolution still needs to be optimized.
Vladimir D:
- T-head just released the new version of toolchain. Tested OpenCV with it, found only a few minor problems. Overall, it works well.
- Helping with 4.9.0 release.
Shiqi Yu:
- Several people attended OpenCV technical meeting. Not many, but next time hope to get more attendants.
- Will do C++ course next year. Want to combine it with OpenCV technical meeting.
Alexander S (opencv.ai):
- Done:
  - Abdurahim is working on object detection API.
  - Working on updating tutorials on using OpenCV for Android.
  - Alexander Panov:
    - Prepared PR Update android OpenCL sample #24715
    - Fixed CMake, gradle, sample and tutorial
    - Successfully build OpenCV on Android with OpenCL and tested sample
    - Updated description in issue Problems with adding OpenCV to Android Studio project #24663 (added solution to include OpenCV)
    - Reviewed findChessboardCorners performance degradation fix for some cases #24558
    - Works with Unify build.gradle generation for Android projects #24686
  - Alexander Smorkalov:
    - Reviewed, tested, helped to resolve issues: ## 24709, 24717, 24715, 24713, 24702, 24700, 24698, 24136, 24576, 24539, 24694, 24666, 24659, 24642, 24689, 24613, 24068, 24629, 24592.
    - Added support for system-wide libspng (#24718), fixed several related warnings (#24716).
    - Prepared several Android related fixes: # 24723, 24699
    - Works on 4.9.0 release preparation.

2023-12-06

Vadim:
- [OpenCV China] transformers acceleration is going on; found some fundamental issues with the current ONNX graph simplifier. will try to find a workaround before 4.9.0 code freeze.
- [OpenCV China] convolution perf tests refactoring is also in progress; found a minor problem in the patch, will hopefully merge it this week
Dmitry M (Intel):
- Q: Age-gender model is not found during test. Looks like a bug. Answer by Alexander S: will look at it.
- Resumed active work on G-API. Updating IE backend in G-API is in progress. Fixed a few bugs when testing G-API + IE backend on OpenCV model zoo.
Vincent:
- Google has just released Gemini (multi-modal vision model): https://www.youtube.com/watch?v=UIZAiXYceBI
- Found one bug in cv.resize() (Python bindings for cv::resize()) where numpy array ([1,1,1,1]) is handled incorrectly.
Gary:
- Continue to work on gaussian splatting
Alexander S (opencv.ai):
- Done:
  - Alexander Panov:
    - Reviewed and tested #24639, #24599, #24616
    - Reviewed and tested ChessBoardDetector::findQuadNeighbors #24605, #24597
    - Refreshed tutorial for Android mobilenet ssd detector (Doc)
    - Fixed and merged PR Add android QR detection sample #24598
  - Abduragim Shtanchaev:
    - Worked on community reported issues with yolo model such as #23977, #24635
    - Prepared API proposal from yolo
  - Alexander Smorkalov:
    - Refreshed Android OpenCL sample. Fixed issues, added to default build config #24639
    - Fixed several Android specific issues. PRs ## 24651, 24641, 24649, 24640
    - Fixed several minor issues in videoio module: PRs ## 24625, 24657
    - Reviewed, tested, helped to resolve issues with ## 24622, 24592, 24611, 24653, 24623, 24647, 24655, 24648, 24630 contrib #3516
  - Rostislav Vasilikhin:
    - Created draft PR with video writing sample: #24592
    - Tutorials for Android development: in progress #24653
  - Alexander Lyulkov:
    - Prepared initial version of Android sample with VideoWriter #24592
    - Fixed issue with static AAR package build
    - Fixed Android sample to save images on sd card with modern Android #24622

2023-11-29

Vadim:
- [OpenCV China] Yuantao has prepared the 1st PR with attention block acceleration. Still quite far from ONNXRuntime (~3x slower).
- [OpenCV China] Wanli has refactored convolution perf cases; FP16 and Winograd are yet to be added.
- Q from Yuantao: it looks like ONNX model zoo (github.com/onnx/models) is going to be replaced with something new. We reference some onnx zoo models in our tests. Need to move them somewhere. We roughly need 16-17Gb of space, which is quite big for internet-shared archive.
Shiqi Yu:
- "non-free OpenCV" idea causes some questions. Need to make a clear statement (re-assure people) about free OpenCV.
- regular seminars (one per month) will be organized by OpenCV team in SuSTech, e.g. on transformers optimization etc.
Vladimir:
- Maksym helped with downloading some ML models used by DNN testing. dl.opencv.org will be used. Let's see if the bandwidth is enough.
- Maksym took a look at PR by Liutong that improves cv::transform() performance (still not fast enough).
- Resampling functionality (for audio) is in progress as well.
- Dmitry has reviewed several PRs.
Gary:
- gaussian splatting implementation for NeRFs is in progress.
Alexander S (opencv.ai):
- Done:
  - Alexander Panov:
    - Review and test PR In-house QR codes decoding #24299, regression was fixed, PR was merged
    - Open and debug issue Crash in the Android camera-calibration sample #24612
    - Prepared PR Add android QR detection sample #24598
    - Reviewed PRs change binarization in findChessboardCorners() #24564 and * findChessboardCorners performance degradation fix for some cases #24558
  - Abduragim Shtanchaev:
    - Added multiple PRs with yolo tests. Yolox, Yolov8, Yolov6
    - Finished preprocessing fix for padding images in tests
  - Alexander Smorkalov
    - Reviewed, tested helped to resolve issues with ## 24596, 24569, 24598, 24611, 24597, 24608, 24604, 24595, 24581.
    - Added workaround for Android samples build with CMake (#24607) . Fixed regression in Android tutorial2 sample (#24600).
    - Created Maven Central account for OpenCV.
    - Tested Android samples with Android 10 and Android 6. Filed bugs #24590,
    - Migrated Android Face Detection sample to DNN API. Fixed ONNX load from memory buffer, added in memory API for FaceDetectorYN class (#24599).
    - Fixed Android sample mobilenet-ssd detector. Added the sample to the regular build (#24616).
  - Alexander Lyulkov
    - Scripts for creating OpenCV Android AAR library has been merged #24456, #24473
    - Works on VideoWriter sample for Android

2023-11-22

Vadim:
- [OpenCV China] Yuantao is still working on multi-head attention acceleration. He advanced with implementation of "batch matmul", based on our accelerated gemm implementation, so the results should be ready quite soon.
- [OpenCV China] the patch from Huawei to opencv_contrib that introduces CANN-based AclMat and some core operations has been finally merged.
- [OpenCV China] Wanli is working on convolution performance tests; the tests appeared to be quite useful and already caught some regressions (see below)
- [OpenCV China] the patch by Zihao to speedup FP16 3x3s1d1 convolution using Winograd has been finally merged as well. Alexander S detected some regressions, so Zihao is now looking at it.
Dmitry:
- need to discuss this "OpenCV HAL" thing.
Vincent:
- 4.9.0-pre is working well, just 1 test failed in the internal tests, but it was not OpenCV fault. So far so good.
Vladimir:
- Provided help with CUDA code paths testing in Linux.
- Reviewed several PRs, including OpenCL-accelerated resize.
- There QR decoder improvements/bug fixes by Dmitry K.
- RISC-V P extension patch has been reviewed.
Gary:
- some updates on the campaign (still is a bit slow)
Shiqi Yu:
- can ask WeChat team whether some comparison can be done between the old and new detector.
- no updates this week.
- new company will likely become OpenCV sponsor soon (the contract is about to be signed).
- another organization in China who supports open-source software movement may start supporting OpenCV soon.
Gary:
- TBD
Alexander S (opencv.ai):
- Done (summary for 2 weeks):
  - Alexander Panov:
    - Review and test with benchmark PR In-house QR codes decoding #24299, regressions were found
    - Checked findChessboardCorners PRs #24654, #24558
    - Build and run on phone OpenCV Android samples, preparing new Android samples with QR code, created docs with tested build flags
  - Abduragim Shtanchaev:
    - Finished writing test for yolox detection model (PR to opened)
    - Writing test for yolov8 detection model
  - Alexander Smorkalov:
    - Reviewed, tested, helped to resolve issues with ## 24299, 24569, 24539, 24561, 24499, 24544, 24551, contrib #3586
    - Fixed exception handing in OpenNI2 VideoCapture backend (#24560)
    - Finalized PR “Updated Android samples for modern Android studio. Added OpenCV from Maven support. #24473”. Prepared new Docker environment, updated CI pipelines.
    - Added AAR package build and test to CI (#24554).
    - Dropped Android Service support (#24575).
- Q by Shiqi: which QR detector is better, the new one (based on AruCO tech) or the one by WeChat team? Can we combine it. A: generally the new QR detector is better, e.g. it can detect QR codes on images with significant distortion

2023-11-15

Vadim:
- [OpenCV China] Yuantao has submitted patch to support HardSwish, LayerNorm, InstanceNorm etc. in CANN backend of DNN. This is to support some new models, including vittrack for object tracking.
- [OpenCV China] Yuantao continues working on transformer acceleration, he is now implementing fused multi-head Attention block.
- [OpenCV China] Wanli reviewed convolution kernels used in various computer vision models, including various flavours of depth-wise convolution. He will submit PR with the respective performance tests for DNN.
- resumed work on the patches by Zihao Mu to accelerate FP16 and INT8 convolution. Hopefully, will finish it this week.
Vladimir:
- finished RISC-V testing on real hardware; added respective information to OpenCV wiki: https://github.com/opencv/opencv/wiki/OpenCV-RISC-V. Lichee Pi supports RVV 0.7.1. Support for RVV 1.0 is coming.
- Maksym fixed a few test failures in 5.x branch: https://github.com/opencv/opencv/pull/24517.
- continue to work on outliers (functions that run very slow on RISC-V with RVV). In particular, fixed color conversion RGB<=>BGR.
Vincent:
- found some potential problems with memory in warp affine/warp perspetive. Will provide fix.
- tried opencv 4.9.0-pre internally, compiles and runs well. Found one issue with 5.0-pre - some Python samples fail.
- maybe, it would be a nice to have a web demo of OpenCV, since we have OpenCV.js and there is even some webasm optimization (among a few other popular projects: https://web.dev/articles/wasm-libraries). For example, ffmpeg has such a demo (see the link in previous sentence).
Shiqi Yu:
- OpenCV training is over. It was about using robotic arm (by Elephant Robotics) + depth camera by Orbbec; Aruco codes have been used for locating hand in space. There have been 45 participants. Wu Jia presented materials, and it was followed by the practical session.
Dmitry:
- working on PR with multi-object tracking support in G-API.
- documentation on G-API is going to be improved as well.
Gary:
- the campaign is going on, but it's a bit slow; ~7% of expected money have been collected so far. Continue to explore possible ways of funding dev team beyond just this campaign.
Alexander S (opencv.ai):
- TBD

2023-11-08

Vadim:
- [OpenCV China] Yuantao did more testing of CANN-based OpenCV acceleration; the patch has been modified by our request, it does not modify Input/OutputArray any longer. Will do further testing in a cloud on Ascend 710 devices.
- [OpenCV China] there was request to port instanceNormalization layer to OpenCL. Tried to do it on ARM-based mac, but found that DNN on macOS does not support OpenCL backend. Will try to re-enable it.
- [OpenCV China] Yuantao continues work on attention block optimization. Looks like we will still need batch matmul.
- [OpenCV China] Wanli continues to explore convolution operations. Will add more perf tests soon.
Shiqi Yu (OpenCV China):
- no updates this week. Preparing for the OpenCV+robotics offline 1-day course (this Saturday).
Vincent:
- tried the latest OpenCV (4.9.0-pre). There are some modifications observed in Python bindings, some weird behaviour of functions that take InputOutputArray, e.g. drawing functions. Will take a look at it in more details and report bug if it's confirmed.
- oss fuzz-based testing of libavif-based image codec in OpenCV has been prepared.
- will prepare next patches for cv::resize(...,INTER_AREA) optimization. Should bring another 20% (!) or so acceleration.
- took the first look at G-API, see how it can be useful for massive image processing.
- suggested to
Vladimir:
- a few fixes in gstreamer-based video i/o backend have been merged.
- audio processing functionality in opencv_contrib: doing some activities, will prepare PR later.
- working on enabling RISC-V+RVV0.7.1 in OpenCV CI (using physical board).
- looking at the outliers in performance curve on RISC-V. Raw data and S-curves (perf scalar vs vector) for x64, aarch64 and riscv64 available here.
Dmitry M (Intel):
- 'image == scalar' in Python seems to behave differently from C++.
- Q: shall we target G-API patches to 4.x or 5.x? A: it depends. bug fixes and optimizations should go to 4.x, major new features should target 5.x.
Gary:
- fundraising on OpenCV 5 has started: https://www.indiegogo.com/projects/opencv-5-support-non-profit-open-source-cv-ai#/
Alexander S (opencv.ai):
- Done:
  - Rostislav Vasilikhin:
    - NaN mask PR #23098: finishing patchNaNs():
      - Backported to 4.x: #24480, perf data: #1118@extra, both merged
      - 64F accelerated: +10%...+20% on 3840x2160, +30% on 640x480
    - finiteMask() vectorized: 1.4-3.x acceleration on F32 data, 1.2-2x acceleration on F64.
    - GSoC triangle rendering: continue following TODO list
  - Alexander Smorkalov:
    - Merged 4.x -> 5.x. Finished RISC-V RVV integration to 5.x and related universal intrinsics cleanup.
    - Fixed several issues ## 24510, 24492, 24511, 24512, 24477, 24474, 24468
    - Reviewed, tested, helped to resolve issues ## 24243, 24479, 24445, 24465, 23098, 24500, 24456, 24495, 24473, 24490, 24397, opencv-gha-dockerfile #24
  - Alexander Panov:
    - Prepared PR Fix contour filtering in ArUco #24479.
    - Prepared and fixed tests, updated docs, refactored code, checked accuracy/performance
    - Improved detection of QR codes with ArucoDetector (+13% in monitor category)
  - Abduragim Shtanchaev:
    - Merged performance tests for Einsum layer. Integrated fastGemm to Einsum implementation (#24509).
    - Works on YoloX tests and support in samples.

2023-11-01

Vadim:
- [OpenCV China] Yuantao has almost finished work on fusing several ops into a multi-head attention block. PR should be ready this week.
- [OpenCV China] Yuantao has done the first round of testing of opencv_contrib's new module with Ascend-based OpenCV acceleration of image processing and basic array processing. We asked author to refactor API a bit.
- [OpenCV China] there is a low-profile activity by Yuantao to enable the new version of TIM-VX backend of dnn in order to support newer-gen NPU (A311D2)
- [OpenCV China] Wanli has almost finished work on SoftMax acceleration and submitted PR: https://github.com/opencv/opencv/pull/24466. There is quite good acceleration, but he needs to refactor the patch to reduce the code duplication.
Shiqi:
- book in Chinese about OpenCV "Image processing and computer vision applications - by OpenCV and Python" https://www.epubit.com/bookDetails?id=UB834406b04afe3 by Yu Shiqi and Wu Jia is complete and published. Jia already started collecting materials for the second edition of the book.
- course on Aruco marker detection & robotic arm control has been prepared and will be held on Nov 11th in Shenzhen. ~50 participants are expected to come.
Gary:
- OpenCV 5.0 fund raising campaign is about to start (today?). Fingers crossed.
- Going to meet with Nerf studio team at Berkley.
- Thinking on how to expand the activities on Nerf and also about possible issues (like data privacy etc.)
Vladimir D:
- performance s-curve (scalability) has been analyzed for some RISC-V optimized OpenCV functions (like color conversion). Found performance gap between scalar version and vectorized version (ran slower) in some functions. It was partly fixed and the work continues.
- Maksym has added extra tests for gstreamer, might be useful to add those tests to OpenCV CI.
- Some first RISC-V hardware with RVV 1.0 started to show up: https://aliexpress.ru/item/1005006164476445.html?sku_id=12000036064507916&spm=a2g2w.productlist.search_results.0.6e1e4a4a0MnIOX
Alexander S (opencv.ai):
- Rostislav Vasilikhin:
  - NaN mask PR #23098: finishing patchNaNs accelerated (SSE2/AVX2):
    - 32F: 3x/3x on 3840x2160, 5x/8.5x on 640x480
    - 64F: 2x/3x on 3840x2160, 2.5x/5x on 640x480
    - TODO: port it to 4.x
    - Vectorized finiteMask() for 32FC3, 32FC4, 64FC3, 64FC1, now accelerating to reach at least 1x
    - Performance tests: fixed, properly measured
    - A lot of minor refactoring
  - GSoC triangle rendering: fixing on my own
    - #24065 was automatically closed after rebase, recreated on my branch as #24459,
    - the same for extra: [#1078@extra], (https://github.com/opencv/opencv_extra/pull/1078) -> #1113@extra
- Alexander Smorkalov:
  - Reviewed, tested, helped to resolve issues ## 24378, 23098, 24458, 24470, 24397, 24405, 24445, 24255, 24461, 24456, 24453, 22571, contrib #3586.
  - Fixed issues ## 24094, 24468, contrib #3584. Reproduced #20202, 20185. Works on the last one.
  - Helped Abduragim with Yolo support analysis.
- Alexander Panov:
  - Worked on the issue Issue “aruco.detectMarkers() fails at a certain angle while the image is perfect #24220”. Prepared new filtering algorithm for ArUco. This algorithm fixes:
  - multiple detection of a single marker;
  - incorrect work with nested contours (issue #24220);
  - detection charuco with big markers (big relative to the board cell)
  - Tested new filtering algorithm for ArUco in synthetic benchmark with marker length rate = 0.8
  - Detected markers up from 95% to 100%
  - Error statistics for the new and the old algorithm (error in 10 pixels corresponds to not detected markers) looks very good.

2023-10-25

Vadim:
- [OpenCV China] Yuantao made some experiments with quantized vittrack - managed to generate quantized model, but it's not quite optimal and is not supported by OpenCV.
- [OpenCV China] Suggest to remove DaSiamRPN and the corresponding interface code from video module (5.x branch).
- [OpenCV China] Yuantao keeps working on optimizing transformer models.
- [OpenCV China] Yuantao started testing the new contrib module with Huawei CANN-based optimization of some basic OpenCV functions: https://github.com/opencv/opencv_contrib/pull/3552.
- [OpenCV China] Wanli continues to work on SoftMax optimization.
Vincent:
- there are some internal feature requests regarding Aruco functionality.
- submitted minor AVIF patches.
- very good effect from the optimized resize(...,INTER_AREA) is observed. Would be nice to integrate it before 4.9.0 release.
- keep working on OSS-fuzz-based testing of OpenCV (avif codec and other parts).
- input from Dmitry: maybe G-API could be useful for massive image processing; see https://github.com/opencv/opencv/blob/4.x/modules/gapi/perf/perf_bench.cpp.
Vladimir:
- still working on CI for RISC-V.
- successfully finished OSoC 2023. The student has prepared several patches to convert many SIMD loops to scalable-SIMD aware loops. All the patches have been merged.
Dmitry (Intel):
- some object tracking PRs in G-API are pending; will try to finalize them.
- more changes before 4.9.0 are expected.
Gary:
- kickstarter campaign for 5.0 is closer to the launch (possibly will happen in November).
- public repository for 3D models? (nerf-based)
Alexander S (opencv.ai):
- Started working on OpenCV 4.9.0; planning to release it in the end of December.
- Done:
  - Abduragim Shtanchaev:
    - Finished ellipses support in Einsum layer
    - Working on fixing problems with YOLOX. Categorized the problems here
  - Alexander Panov:
    - Merged fixes to Objdetect benchmark
    - Issue aruco.detectMarkers() fails at a certain angle while the image is perfect #24220
    - More accurate contours sorting is in progress. Current solution has regression in current tests.
    - SimpleBlobDetector finds incorrect blob center for inverted image #24388. The problem and documentation are being fixed by PR
  - Alexander Lyulkov:
    - Added int32 and int64 support to dnn (currently only for cpu). PR
  - Alexander Smorkalov:
    - Finished 4.x->5.x merge. Filed bug #24437
    - Reviewed, tested, helped to resolve issues with ## 24420, 24429, 24425, 24364, 24440, 24438, 24363, 23098, 24405, 24294, 24322, 24441, 23929, 24371, 20202
    - Works on 4.9.0 backlog grooming.
  - Rostislav Vasilikhin:
    - NaN mask PR #23098: 99% done
      - Dispatched SIMD code
      - Rewrote to scalable intrinsics
      - Rebased, fixed all comments. TODO: measure performance properly

2023-10-18

Vadim:
- [OpenCV China] PR with RAFT model will be merged into opencv_zoo as-is. It takes ~60Mb and takes ~2sec/frame to run in OpenCV, 1.5sec/frame in ONNXruntime.
- [OpenCV China] working on enabling new Khadas SoC board.
- [OpenCV China] looking at ONNXruntime MLAS implementation to do something similar (batched GEMM) in OpenCV.
- [OpenCV China] working on SoftMax optimization
- prepared PR to detect fp16 & bf16 neon instructions and the corresponding options for OpenCV dispatcher.
Vincent:
- Google recently released new version of MediaPipe, it includes some AI demos (like hand gestures recognition) that can be done from within a browser (Google Chrome).
- Resize (INTER_AREA): the 1st patch has been submitted, 15-20% acceleration can be observed (sometimes less, depending on hardware). Another patch will be submitted next (that will bring another 15-20% acceleration).
Shiqi:
- Some of the students from my group are working on Vulkan optimization of DNN.
- Asked Tencent whether they are going to improve their QR code implementation. The answer is positive, they are going to do it in ~3 months.
- There is some conversation with Orbbec is going on.
Alexander S (opencv.ai):
- Done:
  - Alexander Panov:
    - Objdetect benchmark: fixed charuco/chessboard output, add per image histogram, improved accuracy (improved interpolation for warpPerspective, warpAffine, improved rounding)
    - Merged PR add dynamic window in aruco cornerRefinement #24355
    - Issue aruco.detectMarkers() fails at a certain angle while the image is perfect [#24220 (https://github.com/opencv/opencv/issues/24220)
    - the problem has been found and is being fixed (contour filtering needs to be improved)
    - Checked issue SimpleBlobDetector finds incorrect blob center for inverted image #24388, need to update docs
  - Alexander Lyulkov:
    - Created type inference for dnn in 5.x. Working on adding integer support.
    - Added test for cumsum layer inplace case.
  - Alexander Smorkalov:
    - Reviewed, tested, helped to resolve issues with ## 24412, 24420, 24425, 24367, 24405, 24407, 24417, 24382, 24240, 24345, 24324, 24359, 24370, 24363, python ## 906, 903, benchmarks #7.
    - Prepared 4.x->5.x merge #24416

2023-10-11

Vadim:
- [OpenCV China] Yuantao made RAFT model compatible with OpenCV; now we get the same results as ONNXruntime.
- [OpenCV China] Yuantao submitted a few patches with new layers/fixed layers: 1) proper types of const arguments of binary element-wise ops, 2) instance normalization
- [OpenCV China] Wanli works on extending and optimizing SoftMax implementation, based on Ficus NN code.
- working on enabling some modern ARM instructions, like FP16 and BF16 arithmetics, in OpenCV dispatcher. That will make our DNN optimizations more correct (avoid crashes on old hardware & made use of the new hardware capabilities).
Vladimir:
- OpenCV summer of code: there are 3 remaining PRs by the student; the team is now working on finalizing them.
- RISC-V RVV intrinsics API is still not finalized; 0.12 was introduced recently, and it breaks some code. The team (Maksym) will fix it.
Vincent:
- continue working on resize (INTER_AREA). C++ compiler seems to be doing good work on horizontal pass of resize.
Shiqi Yu (OpenCV China):
- travelled to Slovenia for the conference on biometrics. Demonstrated OpenCV, model zoo, face detection & recognition, all running live on a tiny soc with linux. It got a lot of attention.
Gary:
- continue to work on NERFs (Gaussian splatting in particular). The "inference" part can be made really fast. And it's a very perspective research field, could replace traditional computer graphics in the future: https://poly.cam/gaussian-splatting?capture=ac2cd87e-8f34-4dd8-b4e6-623ed1f418bd.
Alexander S (opencv.ai):
- There is interesting pet project: https://github.com/joachimBurket/esp32-opencv. OpenCV running on ESP32 device with 8Mb(!!) RAM.
- Alexander Panov:
  - Reviewed #24364, #24365
  - Fixed PR add dynamic window in aruco cornerRefinement #24355, added statistics
  - Objdetect benchmark: fixed charuco/chessboard output, add histogram output by category and by image
- Alexander Lyulkov:
  - Working on dnn shape and type inference for better supporting different types and for the new engine
- Alexander Smorkalov:
  - Reviewed, tested, helped to resolve issues ## 24322, 23109, 24355, 24233, 24317, 24324, 24325, 24359, contrib #3565.
  - Started work on Python 3.12 support in OpenCV-Python package and some related environment updates: FFmpeg from brew, security updates, etc (#904).
  - Vacation: 2 days

2023-10-04

Vadim:
- No updates this week due to the holidays in China and other projects, except for ...
- the new DNN engine that has been discussed with Alexander S and Alexander L.
Vladimir:
- the OpenCV summer of code student is still working on RISC-V patches: core, imgproc, dnn, calib3d and g-api are covered.
- QEMU-based CI builder for RISC-V config is almost ready.
Vincent:
- resumed work on inter_area acceleration. Achieved ~30% acceleration on 3-channel images.
- playing with the optimal number of threads for different kernels.
- met with JY and Grace in Paris since they attend ICCV 2023.
Gary:
- working with Satya on OpenCV 5 campaign.
- thinking of model zoo for nerfs.
Alexander S (opencv.ai):
- OpenCV 5 kickstarter campaign is about to start.
- Alexander Panov:
  - Reviewed #24354, [#3565] (https://github.com/opencv/opencv_contrib/pull/3565)
  - Prepared and merged add aruco board separation check #24349
  - Incorrect aruco board corners refinement with CORNER_REFINE_SUBPIX #24113
  - Prepared draft PR add dynamic window in aruco cornerRefinement #24355 Tested dynamic window with benchmark. Added statics/histogram to benchmark.
- Abduragim Shtanchaev:
  - Fixing PR comments Ellipses in Einsum.
  - Working on collecting and categorizing problems related to yolo family of detectors
- Alexander Smorkalov:
  - Reviewed, tested, helped to resolve issues with ## 24356, 24325, 24357, 24294, 24333, 24351, 24353, 24322, 24343, 24234, 24323, contrib #3567.
  - Fixed issues ## 24354, 24342, 24344.
  - Extended OpenCV-Python CI to test with Python 3.12.
- Alexander Lyulkov:
  - Fixed CumSum dnn layer: reimplemented algorithm, fixed parser, uncommented tests 24353.
  - Will be working on refactoring shape inference in OpenCV DNN as preparation for DNN engine refactoring.

2023-09-27

Vadim:
- Finally merged 1D/0D patch
- [OpenCV China] Merged patch by Yuantao with more efficient and more complete ONNX Expand implementation
- [OpenCV China] ONNX Gather implementation is submitted but contains some bugs, Yuantao will fix it
- [OpenCV China] Yuantao is now working on multi-head attention implementation and optimization
Dmitry (Intel):
- Q: 4.9.0 release plans? A: it should be out by the end of 2023.
- Q: any deadline for GSoC 2023 videos? A: no replies yet.
Vincent:
- old versions of OpenCV were removed finally from the company repositories.
- will now work on faster resize with area interpolation; Vadim has sent experimental version for evaluation.
Vladimir:
- a few more PRs have been submitted to expand OpenCV support for scalable risc-v intrinsics (some of them have been merged already). The student's Summer of code was extended to OpenCV Fall/Autumn of code.
- still no real hardware with RVV 1.0 support to measure efficiency of the new implementation. Alexander S: OpenCV.ai is working on it.
- Maksym has submitted more patches to OpenCV to improve compatibility with both RISC-V RVV 1.0 and RVV 0.7.x.
Gary:
- Preparing for the Kickstarter program launch.
- Continue working on NERF's.
- Bought quite expensive and very powerful server for OpenCV to train deep models.
- It would be nice to get to enroll more companies into OpenCV membership program. Will probably work on it as well.
Alexander S (opencv.ai):
- Done:
  - Alexander Panov:
    - aruco.detectMarkers() fails at a certain angle while the image is perfect #24220
    - Minimal reproducer was created, debugging
    - Incorrect aruco board corners refinement with CORNER_REFINE_SUBPIX #24113
    - Fixed by setting cornerRefinementWinSize parameter as 0.5 module
    - Need to test in benchmark
    - PR with charuco benchmark. PR was refactored and merged
  - Abduragim Shtanchaev:
    - Finished Ellipses in Einsum.
    - Finished Perf test for Einsum
  - Alexander Smorkalov:
    - Prepared release 4.8.1 for main repo and OpenCV-Python packages.
    - Reviewed, tested, helped to resolve issues with ## 24283, 24309, 24317, 24316, 24037, 24301, 24286, 24303 benchmark #6.
    - Finished work on CI with OpenVINO.
  - Alexander Lyulkov:
    - Finished with Torch7 removal from 5.x DNN.
    - Debugged issue in CumSum layer. Works on the new layer implementation as the current is buggy (#20110).
    - Debugged reduceSum layer (#23221). The bug is not reproducible any more. Works on extended tests.

2023-09-20

Vadim:
- almost finished 1D/0D patch. The patch to main opencv repo has passed CI already, contrib is still in progress; found that cuda* modules are super-slow to compile. Suggest to replace many of the manually written cuda kernels with calls to NVidia NPP: 1) OpenCV already calls some NPP functions 2) it should be more relevant and easier-to-support solution 3) compile speed should increase significantly, the size of binaries should decrease.
- [OpenCV China] GEMM acceleration is finally merged. Yuantao has now started working on multi-head attention acceleration. He also contributed much more efficient implementation of Expand layer.
- [OpenCV China] Wanli has submitted patch with more perf tests for dnn; added a few tests for different versions of Yolo and a few other models. It was also suggested to add models for semantic & instance segmentation, tracking, text detection & recognition, human pose estimation.
- Together with Alexander Smorkalov and Alexander Lyulkov discussed next steps to accelerate refactoring of OpenCV DNN for 5.x. Will have regular meetings to track the progress.
Vincent:
- Updated WebP in OpenCV 4.x. Alexander S: community has requested to update opencv-python with fresh WebP.
- It would be nice to have custom Mat/UMat.
Shiqi Yu (OpenCV China):
- Next week go to Slovenia for the conference on biometrics (https://ijcb2023.ieee-biometrics.org/). Will promote OpenCV and OpenCV model zoo there.
- Several students from Shiqi's group will work on OpenCV. Some of them will work on Vulkan backend. Others will work on point cloud compression. Some more chose FP16 operations (maybe other reduced precision formats as well).
- Visited one interesting startup in Shenzhen: https://www.elephantrobotics.com/en/. There is some very cool (and inexpensive!) stuff there: robotic arms, toy animals etc.
Gary:
- Working on Nerfs; bought Lambda server.
- Talking to some companies on sponsoring OpenCV and running joint projects.
Alexander S (opencv.ai):
- Done:
  - Alexander Panov:
    - PR with charuco benchmark
    - Added chessboard, aruco board, refactoring, added gaussian noise, salt and pepper
  - Abduragim Shtanchaev:
    - Working on corner cases of Einsum layer, namely ellipsis parsing and processing. See full TODO list here.
    - Fix PR comment for first iteration of Einsum Layer
  - Alexander Smorkalov:
    - Finalized 4.x->5.x merge. Fixed conflicts introduced new types support in cv::Mat
    - Reviewed, tested, helped to resolve issues with ## 24283, 24303, 24126, 23929, 24201, 24037, model zoo #194 (VIT Tracker), 24289, 23987, 24132, 24250, 24275, 24074, 24291, 24288, 24278, 24274, 24269, 24260, 24286 opencv_benchmarks #6.
    - Potentially fixed colospace issue in FFmpeg decoding backend waits for feedback. Existing tests update is required (#24290).
  - Alexander Lyulkov:
    - Removed old Torch7 parser from OpenCV 5.x (#24294)
    - Fixed 2 Google oss-fuzz issues (56435, 48452)

2023-09-13

Vadim:
- still working on finalizing 1D/0D patch. Found that OpenCV DNN + CUDA/CUDNN has cyclic dependency on opencv_contrib (just cudev module).
- [OpenCV China] GEMM acceleration is finished. Awaiting for the patch integration, after that Yuantao will work on multi-head attention blocks.
- [OpenCV China] Wanli will work on extending OpenCV DNN perf tests to add more models.
- AR for Alexander: send reminder to GSoC students to prepare the final report/video.
Vincent:
- going to submit PR with fresh WebP version to opencv/3rdparty.
- OpenCV often uses a mixture of integer types (signed and unsigned); sometimes it causes undefined behaviour. Would be nice to fix it.
Shiqi (OpenCV China):
- Some of the students will work on Vulkan backend in OpenCV DNN.
- In total there are ~6-7 students who can work on different OpenCV topics.
Alexander (opencv.ai):
- Abduragim Shtanchaev:
  - Working on corner cases of Einsum layer, namely reduce operation. See full TODO list here.
- Alexander Panov:
  - Prepared PR with charuco benchmark
  - Added rotation, blur, perspective, … transformations. A large test run is being prepared
  - Fixed Aruco dictionary extension issue #24228. PR merged.
  - Prepared PR fix type cast in drawDetectedMarkers, drawDetectedCornersCharuco, drawDetectedDiamonds #24247. PR merged.
- Alexander Smorkalov:
  - Prepared 4.x->5.x merge. Fixed conflicts with scalable intrinsics and new data types in cv::Mat (#24254).
  - Reviewed tested, helped to resolve issues with ## 24243, 24037, 24247, 24245, 24244, 24246, 24178, 24261, 24260, 24263, 24250, 24201, contrib #3558
  - Prepared materials for commercial collaborations with OpenCV.
- Alexander Lyulkov:
  - Updated DNN samples to switch from old Torch7 to ONNX models (24244, #24245).
  - Works on fixes for Google OSS Fuzzer issues #48452 and #56435 in TF parser.

2023-09-06

Vadim:
- GSoC 2023 is over. Out of 12 projects 10 are successful. PRs have been submitted. Now the students should send the final report and video.
- [OpenCV China] Yuantao almost finalized GEMM patch. ARMv7 part still needs to be finished.
- [OpenCV China] Wanli will work on extending OpenCV DNN perf tests that should help to avoid performance regressions in future releases.
Shiqi Yu (OpenCV China):
- OpenCV China is going to launch 1-week course on OpenCV. It will contain some basic information about OpenCV; plus, it may be collaboration with various companies who produce NPUs, cameras etc. The second part of the course may introduce concrete products, how to use them with OpenCV. So, besides education it will let us to build better connection with industry.
  - The course is going to offline, not online (there are already tons of online courses). The first run is expected to be in October; ~50 participants are expected.
Vincent:
- working on fuzzer-based tests; found one issue in image warping.
- would be useful to refactor core headers and, in general, reduce opencv build time
- opencv zoo looks interesting, why don't we advertise it more actively
Vladimir D:
- extending public CI.
- Gary: OpenCV summer of code student who works on RISC-V acceleration of OpenCV could continue work if it adds more value. OpenCV will pay for this extra time.
Dmitry (Intel):
- need to discuss object tracking; probably have some code to contribute to OpenCV on multiple object tracking (this is unrelated to bytetrack).
Gary:
- just got back from a long vacation.
- Satya is preparing proposal on attracting more funding.
- basic framework for NERFs has been created (actually, it's refactored NERF studio) by GSoC and other students. UI is going to be based on natural language requests (using LLMs). Thinking of where to put it, how to package it. Talking to λLabs.
Alexander S (opencv.ai):
- Done:
  - Alexander Panov:
    - Merged PRs fix refineDetectedMarkers #24139,
    - Extended QR code benchmark to use new Aruco-based algorithm with command line #5
    - fixed issue “Incorrect dictionary size returned from cv::aruco::extendDictionary [#24222] (https://github.com/opencv/opencv/issues/24222)”, opened PR 24228
    - checked and commented issue QRCodes decoding without Quirc #24225
    - Working on PR for charuco benchmark
  - Abduragim Shtanchaev:
    - Finished first iteration of the einsum layer about 10 days ago.
    - Working on corner cases of it, namely reduce operation. See full TODO list here
  - Alexander Smorkalov:
    - Reviewed, tested, helped to resolve issues with PRs ## 24128, 24227, 24211, 24139, 24214, 24209, 24216, 24221, 23607, 24215, 24199, 24196, 24204, 24180. Issues #24142 and 24185 are in progress.
    - Identified regression in the test system (TS module). Reverted related patch in #24223.
    - Works on infographics on OpenCV optimization approaches (HAL, Universal Intrinsics, OpenVX, OpenCL, etc).
  - Alexander Lyulkov:
    - Fixed issue with MJPEG encoder. Pull request is merged.
    - Working on converting torch7 (not PyTorch) dnn samples/models to ONNX.

2023-08-30

Vadim:
- GSoC 2023 final evaluation started.
- [OpenCV China] all 5 GSoC projects lead by OpenCV China seem to be successful, PRs have been submitted and are in a good shape.
- [OpenCV China] Yuantao's patch to add broadcast() function to core has been merged
- [OpenCV China] Yuantao has improved "faster GEMM" patch to include AVX2 acceleration. It's in a good shape already but can be improved a little further, which we discussed.
- [OpenCV China] Wanli will update OpenCV's ONNX conformance tests. Some special flag will probably be needed, since there are several hundreds of tests, they take quite a while and it can slowdown our PR testing if we run them each time.
- Vadim is keep working on 1D/0D support in cv::Mat and is also going to provide some draft version of a patch to add detection of FP16/BF16/dotprod instructions and support the respective code paths in OpenCV dynamic code dispatcher.
- [OpenCV China] Wanli can find some folks in SusTech to help to review GSoC PRs about Nerf (the project supervised by Gary)
Alexander S (opencv.ai):
- TBD
Vincent:
- finished migration to OpenCV 4.8.0.
- working on extra fuzzer-based tests.
- would like to get earlier notification about next releases.
Rostislav (opencv.ai):
- GSoC project on triangle rendering is going smoothly, PR has been submitted: https://github.com/opencv/opencv/pull/24065
- would possibly review https://github.com/opencv/opencv/pull/23985 (improvements in octree).

2023-08-09

Alexander Panov:
- Open PR fix charuco checkBoard #24126 to fix #23905
  - now preparing tests
- Checked and confirmed issues #24117, #24127
- Test charuco with synthetic data generator and strong distortion
  - now preparing benchmark
Abduragim Shtanchaev:
- Works on implementation of Einsum layer forward, almost finished.
Alexander Smorkalov:
- Merged 4.x->5.x
- Restored code from Egor Smirnov for ONNX “conformance” tests and dashboard. Reorganized it as dedicated python package, integrated the solution to ONNX infrastructure, added Dockers. See backend-coreboard.
- Reviewed, tested, helped to resolve issues with ## 24133, 24052, 24125, 24122, 24089, 24101, 24108, contrib #3542
- Released OpenCV-Python 4.8.0.76 with typing fix.
Alexander Lyulkov:
- Fixed issue with webcamera on MSMF backend #23937
- Published scripts for creating OpenCV AAR lib
  - Working on minor OpenCV changes for better AAR supporting

2023-08-02

Vadim:
- [OpenCV China] preliminary measurements done by Yuantao after porting ficus' GEMM to OpenCV DNN; it looks like some matrix products in transformer models are accelerated by ~3x. Great results! Now a few tests still fail, Yuantao is working on it.
- [OpenCV China] Yuantao finished broadcast() implementation, also ported from ficus NN. Works well, PR is almost ready (there is one small compile problem in Debug config; it should be easy to fix).
- [OpenCV China] Wanli started to work on reviving ONNX conformance tests in OpenCV DNN. The goal is first to measure the current ONNX coverage and then to significantly improve it in 5.0.
Satya:
- preparing for the OpenCV 5.0 kickstarter campaign
Shiqi Yu (OpenCV China):
- Had very successful OpenCV summer camp: 43 participants. 2 full days. Many things were presented, mostly OpenCV DNN module.
Alexander S (opencv.ai):
- Done:
  - Abduragim Shtanchaev:
    - Has 2 day of vacation
    - Works on implementation of Einsum layer forward, almost finished.
    - Formatted the blog. It is going to be published today.
  - Alexander Smorkalov:
    - Finalized 4.x->5.x merge. Branches are aligned now. The next merge should happen next week. Added Ubuntu 22.04 CI for 5.x.
    - Manually ported calib3d changes. Reviewed, manually adjusted USAC port #24005.
    - Finalized calib3d module split in 5.x for fisheye part #24084. Manually ported Vincent’s patch for stereoRectify #24096.
    - Made experiments with the multi-camera calibration pipeline. Reported status to the GSoC team.
    - Reviewed, tested, helped to resolve issues with ## 24069, 24059, 23900, 23957, 24012, 23689, 24066, 24052
  - Alexander Panov:
    - On sick leave

2023-07-26

Vadim:
- will be on vacation on August 4th-27th.
- Working on refreshing proposal on OpenCV 5.0 DNN module: scope, prioritized list of features, task decomposition.
- No progress with the patches to add new data types and to add 0d/1d support; hope to finalize them within 1-2 weeks.
- [OpenCV China] Yuantao has measured performance of new GEMM: https://github.com/opencv/opencv/pull/23897. We observe noticeable acceleration in some case and yet some other cases become slower. Yuantao will continue working on it.
- [OpenCV China] Yuantao has submitted PR with a popular numpy operation: broadcastTo(): . Need to review. Vadim has suggested to look at Expand implementation in Ficus NN, which implements similar (or even same) operation, but probably in more efficient way. Also, it would be nice to extend our implementation of binary operations to do broadcasting implictly, without creating explicit broadcasted copy of some of arguments.
- [OpenCV China] Wanli will work on reintegration of ONNX conformance suite into OpenCV DNN unit tests. We need to be able to measure ONNX coverage.
Gary:
- Will be on vacation from July 28 till August 6th and then another week in August.
- GSoC student(s) continue work on Nerf, refactoring the code. The results are quite good already.
- Thinking of how to make it possibly a service to generate nerfs out of user-provided images.
Vincent:
- will skip next 3 weeks
- currently working on some bug fixes in 4.8.0: 1) USAC (one test does not pass anymore), 2) Aruco (check for validity), 3) line segment detector (LSD) needs to be fixed.
- Alexander's reply on Aruco (Charuco): Charuco in opencv main is different from opencv_contrib. The boards need to be regenerated and reprinted.
- Working on more generating more fuzz-based tests for OpenCV. Some preliminary results: https://github.com/google/oss-fuzz/compare/master...vrabaud:oss-fuzz:master, https://github.com/opencv/opencv/compare/4.x...vrabaud:opencv:fuzztest. And there are some instructions:
```
python3 infra/helper.py run_fuzzer opencv opencv_fuzz_imgproc@OpenCVDraw.FuzzDrawCircle
git clone git@github.com:vrabaud/oss-fuzz.git
cd oss-fuzz
python3 infra/helper.py build_image opencv
python3 infra/helper.py build_fuzzers --sanitizer address opencv
python3 infra/helper.py run_fuzzer opencv opencv_fuzz_imgproc@OpenCVDraw.FuzzDrawCircle
```
Vladimir (Y):
- Alex and Maksym are on vacation. Doing some minor work on testing OpenCV@RISC-V.
- GSoC student (Liutong) works well, submitted another PR.
Shiqi:
- OpenCV summer camp will be organized in Shenzhen this weekend. DNN module is the main topic of this camp. There will be ~40 participants.
Alexander S (opencv.ai):
- will be on vacation on August 14th-31st
- Done:
  - Alexander Panov:
    - Fixed Wrong shape of Python Aruco dictionary bytesList? #1939
    - Checked and confirmed issues #24011, [#23905] (https://github.com/opencv/opencv/issues/23905), #23810
    - Issue: Charuco board detection is not suitable for calibration? #23873
    - The user reported that it was fixed. But there are problems with other cases (big boards)
    - Prepared blog post
  - Abduragim Shtanchaev:
    - Implemented Preprocessing part of the Einsum layer
    - Working on implementation of forward part
  - Alexander Smorkalov:
    - Reviewed, tested, helped to resolve issues with ## 24043, 23805, 23732, 23922, 24043, 24047, 24001, 24032, 24040, 24029, 24028, 24046, contrib #
    - Fixed several Python tests for CUDA modules (contrib #3535), tuned test parameters to exclude sporadic test failures in DNN with CUDA (#24040). Added python test to CI with CUDA (#107).
    - Prepared CI pipeline and docker environment for DNN and G-API testing with OpenVINO (#109). Reported found issues. Work on failed tests is in progress.
    - Added Java bindings to dnn_superres module in contrib (#3533)
    - Reworked solution for ## 23974 (24055), 23607, 24046.

2023-07-19

All:
- discussion of OpenCV 5 TODO items; how many engineers we will need to finalize it etc.
Vadim:
- GSoC 2023 mid-term evaluations have been finished. 2 people did not pass to the second stage.
- Found einops as extension to einsum; probably should put it into OpenCV
- [OpenCV China] Yuantao has submitted GEMM acceleration patch; now trying to make CI green; added GEMM layer to DNN (CPU implementation for now only)
- [OpenCV China] Yuantao has submitted broadcastTo() function, which mimics the corresponding numpy function or ONNX's Expand operation. Of course, core's add(), multiply() and other binary functions should be extended to support broadcasting internally without doing explicit expensive broadcasting.
- [OpenCV China] GSoC student started working on adding GatherElements and also submitted RAFT model to model zoo (currently using PyTorch in the example, with plans to replace PyTorch with OpenCV DNN
- OpenCV is now built without -ffast-math and in some cases it causes big problems with performance, see https://github.com/opencv/opencv/pull/24001 for example.
Satya:
- having meetings with some potential customers of OpenCV.
Alexander (opencv.ai):
- Abduragim Shtanchaev:
  - Publishing the blog about ChaptGPT and OpenCV issue debugging
  - Einsum layer implementation is in progress.
- Alexander Smorkalov:
  - Dropped Python2 support in 5.x (#23988).
  - Unified and documented calibration flags for pinhole and fisheye camera models (#23990)
  - Unified error logging. Replaced std::cerr usage with OpenCV logging mechanism (#23999)
  - Added Python API tests for CUDA configuration (#107). Skipped unstable tests in contrib (#106).
  - Reviewed, tested, helped to resolve issues with ## 23805, 23936, 24000, 24012, 24001, 24022, 23910, 23991, 24003, 24008, 23931, 24005 contrib #3477
- Alexander Panov:
  - On vacation

2023-06-28

Vadim:
- GSoC: mid-term evaluation is approaching. A few projects are at danger not to pass into the second stage, unfortunately.
- [OpenCV China] GEMM acceleration patch for DNN is almost ready.
- Almost finalized patch to extend cv::Mat with new data types (CI is almost green). It will be 100% finalized and merged after 4.8.0 is released
Vincent:
- Tested 4.8.0 internally. So far so good.
- Are you interested in adding OpenDTAM (or a part of it) into OpenCV? A: it could be an OpenCV summer of code project.
Vladimir:
- working on issue #23577 (MatExpr).
- reviewed several PRs.
- RVV optimization: the student has created the first PR #23885.
Alexander S (opencv.ai):
- OpenCV 4.8.0 is around the corner, should be released this week (along with OpenCV Python package)
- Alexander Panov:
  - PR Fix segfault in QRCodeDetector.detectAndDecodeCurved #23802
  - The parametric splines were added. The parametric splines are not a universal solution. Other fix options are being considered.
  - Prepared PR Fix detect diamonds api #23848 (fix api + test), PR was merged
  - PR add charuco board check #23647 was merged
  - Work on blog post about QR code algorithms in OpenCV is in progress
- Alexander Smorkalov:
  - Prepared OpenCV releases 3.4.20 and 4.8.0. Works on corresponding OpenCV-Python release.
  - Repared PRs ## 23806, 23881, 23863
  - Prepared weekly merge 3.4->4.x
  - Reviewed, tested, helped to resolve issues with ## 23884, 23852, 23878, 23860, 23878, 23884, opencv_contrib #3150, 3475, 3515, 2486,
- Alexander Lyulkov:
  - Works on AAR package with OpenCV for Android.

2023-06-21

Vadim:
- [OpenCV China] GSoC on optical flow: the student and the mentor (Yuantao) have found 2 relatively compact and seemingly efficient models in ONNX format for dense optical flow: RAFT (there are a few variations of it) and DIP. Since they contain some operations that OpenCV DNN does not support now, the first goal is to run them via ONNXRuntime and start working on adding the missing operations to OpenCV.
- [OpenCV China] GSoC on point cloud compression (2 projects): 1 student is on the sick leave, the other student has prepared PR that adds support for colored point clouds I/O.
- [OpenCV China] Yuantao extracted gemm code from Ficus. It runs ~3-4x faster than the original code from BLISLab. The first PR that puts this fast GEMM into OpenCV should be ready by the next week.
- [OpenCV China] All FP32 models in OpenCV model zoo have been converted to FP16. Seem to work well and take ~2x less space.
- Vadim has mostly finished work on adding 1D/0D mat support (https://github.com/opencv/opencv/pull/23473). There are few minor remaining issues, but they will be resolved right before merging this patch to 5.x (after 4.8.0 is released, PR #23718 is integrated and #23473 is rebased on the latest 5.x).
- Vadim has started resumed work, started by Zihao Mu, on adding CV_32U, CV_64U, CV_64S, CV_Bool and CV_16BF into OpenCV. Currently working on extending core module and the respective tests to cover those new types.
Vincent:
- TBD
Shiqi Yu:
- OpenCV China will organize OpenCV summer school at end of July, 2 days long; expected to have ~50 participants.
- Jia has finished book on using OpenCV with Python.
- Gary and Satya attended CVPR 2023. Unfortunately, Shiqi could not get visa.
Vladimir D:
- Finished migration of barcode detector to the main repository; it will become a part of OpenCV 4.8.0.
- Maksym Shabunin started working on improving solvePoly, to make the initialization more robust.
- OpenCV summer of code, RISC-V acceleration project: some PRs to OpenCV are expected soon.
Dmitry Matveev (Intel):
- Asked about handling security issues when using OpenCV in production: security@opencv.org is the channel to report such issues.
- Anatoly has found some problems in Python bindings for G-API, but it will be addressed after the release.
- There are performance issues with G-API-related GSoC projects, will try to communicate to the students.
Alexander S (opencv.ai):
- Abduragim Shtanchaev:
  - on vacation
- Alexander Panov:
  - Checked issue cv2.findChessboardCorners() running endlessly on certain images #23558
    - The problems in the algorithm were found and described in the issue
  - Checked issue Doesn't correctly recognize some QR #21417
    - The problem in the algorithm was found, decodeMulti() and decode() work differently in QRCodeDetector
  - Wrong shape of Python Aruco dictionary bytesList? [#1939](https://github.com/opencv/opencv_contrib/issues? q=is%3Aissue+is%3Aopen+1939)
    - described problem in the issue, docs update is being prepared
  - Reviewed PR Aruco MIP dictionary #23785 was merged
  - Prepared PR Fix segfault in QRCodeDetector.detectAndDecodeCurved #23802. Added test, added “hack” solution. Works on splines parametrization now.
- Alexander Smorkalov:
  - Reviewed, tested, helped to resolve issues with ##23799, 22798, 23825, 23812, 23807, 23796, 23786
  - Extended JS bindings for bar code detector (23819). Fixed bar code detector build without dnn (23819).
  - Done pre-release tasks for 4.8.0 and 4.3.20: merged 3.4->4.x (23827), incremented library version (23831, 23833), updated FFmpeg wrapper for Windows (23841), covered several documentation issues ## 23832, 23829,
  - Tested and merged several patches for Python types improvement. Tested compatibility with newly released Numpy 1.25.0.
- Alexander Lyulkov:
  - Joined the team. Start working on Maven Central package for OpenCV for Android.

2023-06-14

Vadim:
- [OpenCV China] still trying to run RAFT optical flow model; found ONNX model, will make ONNXruntime-based demo and then extend OpenCV to support such model.
- [OpenCV China] continue working on fast GEMM for OpenCV DNN. It's not black-box GEMM, but rather specially optimized matrix multiplication engine that preprocesses constant matrices outside of the main processing loop.
- Continue working on 1D/0D mat support: https://github.com/opencv/opencv/pull/23473
Vincent:
- finished libavif; PR has been merged. libavif support will be off in 4.8.0, but will be changed to "ON" in 4.9.0 and 5.0.
Vladimir D:
- barcode review is almost ready for integration, should be merged before 4.8.0.
- OpenCV summer of code student (Han Liutong) continues to work on automatic LLVM-based code transformation to support RISC-V.
Dmitry M (Intel):
- more work to add/improve OpenVINO 2.0 support in OpenCV has been done: https://github.com/opencv/opencv/pull/23796. Will likely be merged into 4.8.0.
- GSoC student is traveling this week, will resume work on the project in the coming dates.
Alexander S (opencv.ai):
- Done:
  - Abduragim Shtanchaev:
    - Checked forward crash issue; Worked on reproducing oss-fuzzer problem
    - Working on writing blog on ChapGPT usage for OpenCV debugging
  - Alexander Panov:
    - Reviewed PR Added Aruco-based QR code detection method to python sample #23789
    - Reviewed PR added Aruco MIP dictionaries #23785
    - Fixed Segfault in QRCodeDetector.detectAndDecodeCurved #22892
    - PR generic graphical code interface #23758 was fixed and merged
  - Alexander Smorkalov:
    - Fixed several issues in JavaScript bindings generator. Restored bindings for old QR Code algorithm, added bindings for the new one with Aruco (## 23725, 23769, 23790, 23797)
    - Added Python types information (#853). Checked with MacOS x86 and Linux. Works on incoming feedback from the community.
    - Reviewed, tested, helped to resolve issues with ## 21959, 22947, 23596, 23766, 23786, 23666

2023-06-07

Vadim:
- [OpenCV China] no progress on optical flow project (3 models have been selected).
- [OpenCV China] there was initial meeting with Vincent regarding possible integration of Draco integratio
- There is a strong intention to release OpenCV 5 this year. Need to update the list of tasks and prioritize it.
Vincent:
- met with GSoC folks who work on point cloud compression.
- finished libavif support, could possibly be put into OpenCV 4.8.0
- can the new OpenCV QR detector detect these? :) https://arstechnica.com/information-technology/2023/06/redditor-creates-working-anime-qr-codes-using-stable-diffusion/
Shiqi:
- looking for more sponsors for the upcoming OpenCV competition.
- CVPR conference: looks like Shiqi won't be able to attend it, since no visa is received, just like most of mainland China citizens who applied for visa for CVPR 2023. quite sad.
- some companies are interested to bring in support for the new generic camera protocol (https://en.wikipedia.org/wiki/GenICam) to OpenCV. Any interest from our side?
Dmitry:
- some G-API PRs have been merged, thanks to Alexander & Alexander for assistance.
- GSoC projects are in the initial stage, so far go smoothly.
- would be useful to have API to compute shapes of output tensors in OpenCV DNN without running model.
Gary:
- in total 6 people are working on 1) ChatGPT in OpenCV and 2) NERF.
Alexander S (opencv.ai):
- Done:
  - Alexander Panov:
    - Add detect qr with aruco #23264 - was fixed and merged
    - Prepared and merged PR #23748 to fix issue Generation of getter and setter #23700
    - Fixing bindings issue #3506, 23753
    - Prepared PR generic graphical code interface #23758
    - Fixing issue Segfault in QRCodeDetector.detectAndDecodeCurved #22892
  - Abduragim Shtanchaev:
    - Fixed other issue related to FasterRCNN inference (https://github.com/opencv/opencv/pull/23746)
    - Investigate why FasteRCNN can not be used for inference in OpenCV. Problem related to TopK layer (not implemented for numerous reasons)
    - Investigated issues related to Mul layer.
  - Alexander Smorkalov:
    - Updated Javascript bindings for Aruco API, added tests (#23725)
    - Added Python type stubs to Python package (#853)
    - Prepared CI environment and pipeline with Ubuntu 22.04 (#101)
    - Handled income security issues, fuzzer issue and issues found by Coverity
    - Reviewed, tested, helped to resolve issues with ##23596, 23688, 23729, 23751, 23750, 23748, 23732, 23756, contrib #3453

2023-05-31

Vadim:
- There is a strong intention to release OpenCV 5 this year. Need to update the list of tasks and prioritize it.
- [OpenCV China] GSoC projects are on-going, more meetings with students have been conducted.
- [OpenCV China] Zihao Mu has left the team, but is still finalizing his PRs on FP16 convolution acceleration: https://github.com/opencv/opencv/pull/23654.
- [OpenCV China] Porting faster GEMM from ficus NN has been started.
Shiqi:
- The competition is going to be announced at the beginning of June. Now we are discussing with some potiential sponsors. Hopefully it all will be decided in one week.
- GSoC: Dmitrij is working on a compression algorithm TPCC (https://arxiv.org/abs/2303.16716) (the plan is here: https://github.com/opencv/opencv/issues/23704), and Yuhang team is working on compression by OCTree and Point Cloud encoding.
- there are some delays with visa to Canada to attend CVPR. Hopefully, it will be resolved soon.
Vincent:
- continue to work on libavif, the exif part.
- still waiting for reply regarding initial shake-hands meeting with Draco devs and GSoC students working on point cloud processing.
- going to take a look at and maybe refresh libwebp support: https://repology.org/projects/?search=libwebp
Vladimir:
- Maksim is working on PR to transfer barcode from contrib to main.
Dmitry M (Intel):
- finalizing PRs for 4.8, including support for OpenVINO 2.0.
Gary:
- started talking to GSoC students; actually, more students participate in this project, not only GSoC students.
- going to take lambda server to speedup experiments.
Alexander S (opencv.ai):
- Done:
  - Alexander Panov
    - Add detect qr with aruco #23264 - fixing issues from Alexander Alekhin fixed objc binding problems
    - Probably a bug in aruco::getBoardObjectAndImagePoints #2921 - checked, updated sample to new API, issue was fixed by prev PR
    - Generation of getter and setter #23700 - reproduced, fixing
    - Aruco's CharucoBoard/GridBoard constructor Python signature missing in docs #23133
    - The problem with the generation of docs for python constructors was found.
  - Abduragim Shtanchaev:
    - Fixed issue related to subgraph simplification in FasterRCNN
    - Fixing other issues related to FasterRCNN inference (split, concat)
  - Alexander Smorkalov:
    - Prepared 4.x->5.x merge for all repos. Ported several fixes 4.x->5.x and 5.x->4.x to reduce merge conflicts.
    - Handled several incoming issues and PR for OpenCV Python (PRs ## 839, 838, 837). Hacked buildissue with Scikit-build 0.17.3 and newer (#852).
    - Tested and, finally, merged Python type stubs generator in main repo. Tuned OpenCV python pipelines accordingly (#849).
    - Updated OpenJPEG2000 to 2.5.0 (#23682)
    - Fixed fps estimation in FFmpeg backend in cv::VideoCapture (#23692)
    - Reviewed, tested, helped to resolve issues with ## 23264, 23698, 23688, 23691, 23664, 23628

2023-05-24

Vadim:
- OpenCV China: This is the final week when Zihao Mu is official member of OpenCV. He contributed FP16 Winograd convolution PR, which achieves quite good acceleration. Another INT8 convolution patch is in progress.
- OpenCV China: Yuantao Feng is going to meet with GSoC student this week. He will work on optical flow models; we reviewed possible options.
- GSoC: initial meetings have been organized with several students.
Vincent:
- still need to take a look at the new implementation of resize, why it's slower on some devices (e.g. Jetson).
- Vadim's AR: send e-mail to Vincent and mentors and students who work on point cloud compression (related to possible Draco integration)
Dmitry M (Intel):
- Q: what's the status of Aruco in OpenCV? E.g. any speed improvements? A: no speed improvements, only bug fixes and improvements in accuracy.
- some internal low-profile work is on G-API, no concrete deliverables/milestones yet.
Vladimir D:
- PR with barcode performance test is finalized and merged.
- preparing PR to migrate from contrib to main. Plan to integrate it to OpenCV 4.8.0.
- GSoC student keeps working on porting SIMD loops to "Scalable SIMD" (compatible with RISC-V RVV).
Alexander S (opencv.ai):
- V. Ponomarev has left OpenCV team
- Abduragim Shtanchaev:
  - Fixed null pointer dereference with corrupted TF model #23645
  - Debugged issue “LSTM layer with batch_firs=True fails #23602” - False alarm.
  - Debugging dnn issue # 21634 with FasterRCNN
- Alexander Panov:
  - Addressed review remarks for QR code detection algorithm from Alexander Alekhin
  - Fixed charucoBoard detector issue with wrong board size #23647.
  - Debugs diamond board regression after fix #23647.
- Alexander Smorkalov:
  - Fixed issues ## 21035, 21006, 21604, 23590. Proposed alternative solution for #12343. Debugged #22218. Reported warning is not a bug, but I do not see a simple solution to satisfy the compiler.
  - Reviewed, tested, helped to resolve issues with ## 23363, 19530, 23612, 23371, 23525, 21959, 23264, 23575
  - Prepared regular 3.4 -> 4.x merge. Works on 4.x -> 5.x.

2023-05-10

Vadim:
- GSoC 2023: Google gave us 12 slots, now mentors should contact students.
- OpenCV China: FP16 patch is ready for the final review
- OpenCV China: Continue work on transformer acceleration
Vincent:
- in 1d/0d patch maybe it makes sense to add some more tests (see comments in the PR: https://github.com/opencv/opencv/pull/23473)
- avif support via libavif can be considered as done (rgb/rgba/gray are supported, 8/10/12 bits per channel are supported): see https://github.com/opencv/opencv/pull/23596.
  - two things need to be fixed: 16 bit support and automatic rotation
Shiqi:
- contacted GSoC students. Going to have weekly meetings.
- plan to attend CVPR 2023 with other opencv.ai folks.
- Zihao is leaving OpenCV China. He should still be able to mentor GSoC tracking-related projects.
Gary:
- need to fulfil GSoC-related papers
Alexander S (opencv.ai):
- currently on vacation
- need to round up the list of PRs & issues for OpenCV 4.8.0
- still analyzing behaviour of the new resize from Vincent.
- Done:
  - Abduragim Shtanchaev:
    - Working on implementing layout attribute for LSTM layer
  - Alexander Smorkalov:
    - Handled PRs and issues #23557, 23571, 23584, 23555, 23580, 23558, 23577
    - Made backlog grooming for 4.8.0
  - Alexander Panov:
    - Debugs issue Charuco detector not detect charuco board 2X15 size #23435
    - Fixing bug with wrong size charuco board detection
    - PR “Qr detect with aruco #23264” objective-c binding problem was fixed by author, original PR was rebase

2023-04-26

Vadim:
- GSoC 2023: still waiting for Google's decision on slot allocation
- currently on vacation (hope to finalize 1D/0D Mat patch)
Vincent:
- cv::resize(..., INTER_AREA) is finalized (https://github.com/opencv/opencv/pull/23525), ready for review. Some further optimization is still possible.
- AVIF support via libavif is in progress
- another method to support AVIF is via ffmpeg.
Vladimir D:
- Alexander A is on vacation
- Improved documentation about rotated rect has been merged
- Trying to resolve old issue with ffmpeg on Windows (https://github.com/opencv/opencv/issues/20114)
Alexander (opencv.ai):
- Done:
  - Vladimir Ponomarev:
    - Fix new notes for stereo_calib.cpp and interactive_calibration app
  - Alexander Smorkalov:
    - Reviewed, tested, helped to resolve issues with #23479, 23342, 13879, 22614, 23525, contrib # 3477, 3410
    - Fixed CUDA accelerated VideoReader properties reporting in contrib #3485
  - Abduragim Shtanchaev:
    - continue to work on LSTM implementation in OpenCV DNN; will look at GRU after that.
    - on vacation
  - Alexander Panov:
    - Refactored QR code detector API to extract Aruco-based implementation as separate class
    - Finished with matchImagePoint fix for Python. Merged.
    - Debugs issue Charuco detector not detect charuco board 2X15 size #23435
    - on vacation

2023-05-03

Vadim:
- Finalizing the selection of GSoC 2023 projects; almost done
- [OpenCV China] Mu Zihao:
  - Mu Zihao: prepared PR with revised text detection API
  - finalized the first Vulkan backend refactoring PR - suggested to merge as soon as CI is green
  - finalized FP16 support patch - there is some slowdown currently, but with FP16 Winograd (in the next PR) it should become faster.
- [OpenCV China] Feng Yuantao:
  - preparing materials (slides) for the subsequent work on Ascend backend.
  - ran SAM using OpenCV, will likely put it to opencv model zoo soon (in compressed form).
Vincent:
- patch with AVIF support is in progress. There are 2 options: a) ffmpeg (VideoCapture can read .avif images already), b) libavif.
- new optimized resize(..., INTER_AREA) is almost ready (https://github.com/opencv/opencv/pull/23525). Now need to fix the remaining tests.
- it would be useful to have 1D vector type conversion "intrinsic" in OpenCV HAL; will submit the corresponding feature request.
Vladimir D:
- working on performance tests for 1D barcode in opencv_contrib. Maybe someone from Y team can port 1d barcode from contrib to the main repository.
Satya:
- some discussions with legals
- reimbursment info related to the past OpenCV forum is now being collected.
- OpenCV crowd-funding through github needs to be advertised.
Shiqi:
- Need to prepare CVPR 2023 posters and prepare some demos
Gary:
- reviewed ChatGPT & NERF proposals.
Alexander S (opencv.ai):
- 3D visualization is still in our TODO list
- Done:
  - Abduragim Shtanchaev:
    - Checked why conformance tests fail for LSTM. Opened issue #23542
    - Found issue related to cuDNN 8.x not using CUDA backend #23505, #23455
    - Covered LSTM with more tests #23545
    - Working on implementing LAYOUT attribute for LSTM
  - Alexander Panov:
    - PR “Fix aruco module CORNER_REFINE_CONTOUR parameter gets skipped #23495” was reviewed and merged
    - PR "Include pixel-based uncertainty in ArUco marker detection #23190" was reviewed
    - PR “Qr detect with aruco #23264” fixed review remarks
    - PR “Fixed PR charuco matchImagePoints #23138” fixed review remarks
  - Vladimir Ponomarev:
    - Fix notes for calibration.cpp, stereo_calib.cpp and interactive_calibration app
  - Alexander Smorkalov:
    - Introduced tests for Python apps (apps folder) in main repo and CI (23363, pipelines #93).
    - Merged 3.4 to 4.x branch and resolved conflicts.
    - Reviewed, tested, helped to resolve issues with #23486, 21118, 23475, 23531, #23528, 22750, contrib #3477

2023-04-19

Vadim:
- Reviewing GSoC 2023 submissions: we've got 118 proposals. Did the first pass, will take a look at the better proposals for another time to select the best ones.
- [OpenCV China] improved blobFromImages with extended functionality is ready and will be merged.
- [OpenCV China] continue working on OpenCV Zoo benchmark, for now the performance results are questionable, need to make sure we built OpenCV properly, configured test environments properly and measure the performance correctly.
Shiqi Yu:
- no updates so far
Vladimir D:
- taking a look at some very old PR, such as cv::reduce extensions.
- checking now well fresh GCC compilers support RVV (it seem that the newest "scalable" backend for RVV is supported well).
Vincent (welcome back!):
- One OpenCV version is now used in some important apps (!)
- Resize w. INTER_AREA would be useful to optimize.
- Virtual dummy 3d and calib could be added to 4.x to simplify further migration to 5.x (and maybe vice versa).
- Text drawing causes some regression tests to break
- AVIF support in OpenCV is probably coming.
Alexander (xperience.ai):
- Done:
  - Abduragim Shtanchaev:
    - Fixes related to LSTM layer dynamic input initialization PR #23475
    - Checked if LSTM supports various batch and length sizes. Added tests #23501
    - Checked if GRU layer is working.
    - Working on fixing conformance test that are running for LSTM
  - Alexander Panov:
    - PR “Add IoU metric to QR code benchmark (#4)” was fixed and merged
    - PR “Fix build regression in Quirc #23484” was merged
    - PR “Update charuco_detector.hpp #23436” was reviewed and merged
    - Qr detect with aruco #23264 fixed review remarks
  - Vladimir Ponomarev:
    - Fixed last comments on cv::VideoCapture patch
    - Added ChAruco pattern into calibration.cpp
    - Added ChAruco pattern into stereo_calibration.cpp
  - Alexander Smorkalov:
    - Fixed Python tests lookup in CI (#94). Added dependencies needed for gen_pattern.py tests (#17, #93).
    - Reviewed, tested helped to resolve issues with ##23501, 22750, 23097, 23486, 23492, 23436, 23363, 23055, 23264 contrib #3468, 3470, 3462, 3465

2023-04-12

Vadim:
- All the proposals have been collected and will be reviewed this week.
- Resumed work on adding 0D/1D Mat support: https://github.com/opencv/opencv/pull/23473
- [OpenCV China] Yuantao Feng. repaired OpenCV CI; now it should work correctly. Looked at Facebook's SAM model. OpenCV currently does not support it due to missing layers. Now fixing it
- [OpenCV China] Zihao Mu. FP16 convolution is still in progress, there are some regressions that need to be fixed.
Alexander S (xperience.ai):
- Done:
  - Abduragim Shtanchaev:
    - Fixed LSTM layer dynamic input initialization #23475
    - Checking if LSTM is able to work with inputs of different sequence length
  - Alexander Panov:
    - PR “Fix flipped image in documentation #3469” was reviewed, fixed and merged
    - Added IoU metric to QR code benchmark (#4) - works on review comments
    - Create PR “Fix build regression in Quirc #23484”
    - Reviewed PR “Include pixel-based uncertainty in ArUco marker detection #23190”
    - Fixed review remarks Qr detect with aruco #23264
  - Alexander Smorkalov:
    - Reviewed, tested, helped to resolve issues with #23484, 23363, 23264, 22660, 23472, 23447, 23476, 23433, 23441, 23446
  - Vladimir Ponomarev:
    - Finished with charuco support pattern generator #23363
    - Works on charuco support in interactive calibration tool (#23479) and calibration related samples

2023-04-05

Vadim:
- Discussed OpenCV 5 functionality @ OpenCV forum. It's planned to make OpenCV 5 smaller, faster, more relevant. Follow-up tickets will be created and discussed.
Vladimir D:
- Maksym has submitted fix for RVV in Clang 16.
- Trying to resolve problems when using the latest available RISC-V+RVV compiler from T-head
Shiqi Yu:
- now traveling
Alexander S (xperience.ai):
- Done:
  - Abduragim Shtanchaev:
    - Checked inference of LSTM layer for correctness. Fixed a bug related to the output of the LSTM layer
    - Checking why some tests are not passing after fixes
    - After LSTM plan to revise GRU
  - Alexander Panov:
    - PR Aruco/Charuco test case fixes for floating point for loops #23271 was reviewed, fixed and merged
    - Qr detect with aruco #23264
    - Add intersection_over_union metric, created new PR to opencv benchmark
    - Update QR statistics, in 190 images the QR detection became better.
    - Fixing performance tests now
  - Alexander Smorkalov
    - Participated in F2F summit. Prepared materials
    - Debugged, tested, heped to resolve issues: #23448, 23363, 23442, 23446, 23321, 23451, 23449, 23419, 23418, 23410
  - Vladimir Ponomarev
    - Added options for custom dictionary usage in boards generator. Moved dictionary generator from contrib to main repo. Replaced pre-defined dictionaries by compressed to reduce patch size. Documented usage.
    - Tested #18180. The issue is not reproduced any more.
    - Works on charuco support in existing calibration samples and interactive calibration.

2023-03-29

Vadim:
- [OpenCV China] Yuantao Feng continues to work extending CANN backend to support more operations and models in order to be able to run the whole OpenCV deep learning benchmark on it.
- [OpenCV China] Zihao Mu has resumed work on enabling FP16 convolution in OpenCV DNN (based on Ficus NN code). The regular im2row-based convolution is enabled now, Winograd is still TBD, but the results are promising already: On M1 chip we get 25% speedup on Resnet-50, ~30% speedup on MobileNet: https://github.com/opencv/opencv/pull/22275. The PR is still being polished to successfully pass CI tests.
Shiqi Yu:
- TBD
Vladimir D:
- Dmitry Kurtaev is adding int8 support into TFLite importer
- Maksym Shabunin is fixing RVV backend of universal intrinsics according to the latest changes in clang 16
- Alexander Alekhin keeps reviewing PRs
Alexander S (experience.ai):
- TBD

2023-03-22

Vadim:
- [OpenCV China] Yuantao Feng has greatly extended opencv_zoo benchmark with more configurations and more devices (even more devices are coming): https://github.com/opencv/opencv_zoo/blob/c609d377450508d27cb0acc52b85eff3a6bc1841/benchmark/README.md.
- [OpenCV China] Zihao Mu is finishing work on Vulkan backend, currently there is one slight problem with deinitialization on our Windows test machine. On Mac with M1 and Molten everything works well. He also plans to port FP16 convolution code from Ficus NN that should bring noticeable acceleration on ARM (ARM v8.2 or later; unfortunately, RPi is not in this list).
- [OpenCV China] Zhong Wanli has prepared PR into opencv_zoo with the efficient human pose estimation model (from MediaPipe).
Shiqi Yu:
- negotiated about possible alternatives to github
Vladimir:
- 0.10 => 0.11: each intrinsic name is now prepended with _riscv. Also, continue experiments with fresh compilers that support RVV.
Alexander S (experience.ai):
- Abduragim Shtanchaev:
  - Getting acquainted with DNN module structure
  - Working on eliminating problem of LSTM layer initialization
- Alexander Smorkalov:
  - Fixed all remarks to multi-camera calibration PR. Plan to merge it today after yet another test round with real data
  - Fixed OpenCV-Python package build issue
  - Reviewed, tested, helped to resolve issues with # 23264, 23363, 23380, 23275, 23019, 23250, 23348, 23344, 23325, 23380, 23377, 23196, 23292, 23358 contrib # 3453
- Alexander Panov:
  - Checked and reviewed PR Added QR_Code data flip support, flip and retry after first ECC failure #23275
  - Add detect qr with aruco #23264
  - Timing pattern checks and metrics were updated and improved.
  - Fixed bugs with markers thinning (partially).
  - One of the loops was reduced, performance was improved.
  - Parameters and thresholds were adjusted.
  - Refactoring + adding new tests now.
- Vladimir Ponomarev
  - Finished charuco board generator based on svgfig
  - Finished tests of charuco board generator
  - Working on documentation
Gary: *

2023-03-15

Vadim:
- [OpenCV China] Yuantao Feng has achieved very good preliminary results in vision transformers acceleration. The speedup is ~4x, but the code is still ~20% slower than ONNXRuntime, with clear path how to eliminate the difference completely. The PR should be ready next week.
- [OpenCV China] Zihao Mu has achieved 4x acceleration in Vulkan backend (https://github.com/opencv/opencv/pull/23349). Vulkan version is still somewhat slower than CPU, so the work will continue.
Shiqi Yu:
- TBD
Alexander S (experience.ai):
- Org:
  - Abduragim Shtanchaev joined OpenCV team
- Done:
  - Alexander Panov:
    - Fixed and merged PR add python charuco sample #23139
    - Checked and reviewed PR Added QR_Code data flip support, flip and retry after first ECC failure [#23275] (https://github.com/opencv/opencv/pull/23275)
    - PR Remove separator between trackbars #23261 was tested and approved
    - Testing and fixing QR code detect with ArUco algs
  - Abduragim Shtanchaev:
    - Install OpenCV from source on Mac and Linux
    - Getting introduced with general documentation and on DNN specifically
  - Alexander Smorkalov:
    - Debugged and fixed C++ sample for multi-camera calibration. Raised concerns on API, proposed alternatives
    - Reproduced OpenCV-Python built issue for platforms, where pre-built wheel is not provided. Fixed the bug (#820)
    - Reviewed, tested, helped to resolve issues with ## 23139, 23357, 23348, 23291, 23354, 23341, 23276, 23325, 23328, contrib 3453
  - Vladimir Ponomarev:
    - Finished PR with flann memory leak
    - Finished PR with obsolete video backends flags in openCV 4.x
    - Create charuco board generator based on svgfig
    - Start testing of charuco board generator

2023-03-08

Vadim:
- [OpenCV China] Yuantao Feng has submitted PR with improvements and bug fixes in CANN backend
- [OpenCV China] Zihao Mu continues to work on Vulkan backend for OpenCV DNN. GEMM has been significantly accelerated, but it's still slower than CPU. Further investigation is needed.
Shiqi Yu:
- Currently using OpenCV for courses - want to integrate OpenCV with some robot arm or NPUs
- Zheng Wangli from OpenCV China is helping to organize OpenCV environment for such educational "vision in robotics" projects.
- CVPR 2023 will happen in Canada, June. Wish to participate there (have a booth?)
Dmitry M (Intel):
- Talked to several teams who may be interested to cooperate with OpenCV. Some initial connections between teams have been established already.
- Support for multiple video streams in G-API is being designed now.
Gary:
- tried ChatGPT to write some OpenCV code - looks promising :)

2023-03-01

Vadim:
- GSoC 2023: discussion of the possible projects has started in the google group.
- [OpenCV China] Yuantao Feng temporarily switched from the task of optimizing vision transformers in OpenCV DNN to updating CANN backend to handle more models correctly. The patch is in progress.
- [OpenCV China] Zihao Mu continues to work on Vulkan backend for OpenCV DNN. So far, no significant progress in speed.
- [OpenCV China] Zheng Wangli looked at 2 models from MediaPipe for human detection and human pose estimation. The models work very fast on a laptop using CPU, e.g. people detector runs at 20fps with OpenCV. The models are also very compact 11Mb and 5Mb respectively. Looks like we could try to further compress them (e.g. to FP16) and migrate to OpenCV model zoo.
Shiqi Yu:
- idea for OpenCV Python: would be useful to have CUDA-accelerated distro of OpenCV Python.
Vladimir D (Yadro):
- TFLite parser has been merged
- Alexander Alekhin continues to review various patches and fix bugs here and there
- OpenCV master version can now be built with master version of T-head GCC (that supports RVV 0.7.x).
- Maksim Shabunin reviewed PR https://github.com/opencv/opencv/pull/23260 with some refactoring for GTK support. And once this will be merged, it should open the door for GTK4 support
Gary:
- Continue to play with Nerf models.
- Also played with Aruco tags recently.
Alexander S (xperience.ai):
- Done:
  - Vladimir Ponomarev:
    - Reworked fix for potential resource leak in Flann #23276
    - Checked PR #23300. (White balance in DShow)
    - Fix issues with old flag (CAP_OPENNI flag) in OpenCV contribute, (5.x and 4.x branches)
    - Start working on the charuco board generator.
  - Alexander Smorkalov:
    - Reworked documentation for multi-camera calibration (#22363)
    - Fixed review notes in PR for multi camera calibration (#22363)
    - Revied, tested, helped to resolve issues with #23276, OpenCV-Python ## 808, 809, 810
  - Alexander Panov:
    - Seek leave

2023-02-22

Vadim:
- OpenCV has been accepted into GSoC 2023! Congratulations to the team! :)
- [OpenCV China] Yuantao Feng continues to improve vision transformers performance using faster GEMM.
- [OpenCV China] Zihao Mu continues to work on Vulkan backend for OpenCV DNN.
- [OpenCV China] Zheng Wangli just finished work on palm pose estimation and prepared technical report.
Shiqi Yu:
- experimenting with robotic arm, trying to make it play "paper-rock-scissors" game using just added to opencv zoo model to estimate palm pose
- had a meeting with organizers of Chinese University Student Computer Design Contest; OpenCV China is going to participate.
Vladimir D (yadro):
- It appears that the latest T-head compiler (https://github.com/T-head-Semi/xuantie-gnu-toolchain) is able to compile RVV 1.0 universal intrinsics backend in OpenCV, and such a compiled binary can run successfuly on T-head Xianti C906 CPU (RVV 0.7.1). and pass all tests. The PR to OpenCV has been submitted and merged (https://github.com/opencv/opencv/pull/23246)
Stefano F:
- There is interesting end-to-end compiler from StableHLO to anything: https://github.com/openxla/iree
Alexander S (xperience.ai):
- Investigated OpenCV camera calibration tests. Upon detailed inspection, the quality of calibration is questionable, and the quality of tests is questionable. Need much better tests first of all, and then maybe need to tune the camera calibration parameters.
- Alexander Panov:
  - Fixed remarks in “Fixed PR charuco matchImagePoints #23138”
  - Finished initial version of new QR Detection algorithm. Submitted PR #23264
- Alexander Smorkalov:
  - Prepared OpenCV-Python 4.7.0.72 bug fix release
  - Reviewed and fixed PR for multi-camera calibration (GSoC project). The PR is ready to merge.
- Vladimir Ponomarev:
  - Fixed potential resource leak in Flann #23276
  - Revised cv::VideoCapture backends. Removed constants for back-ends that do not exist in 5.x. Prepared PR with stub code for 4.x to not break compatibility.
  - Fixed trackbar layout issues on Windows #23261

2023-02-15

Vadim:
- [OpenCV China] Yuantao Feng: vision transformers acceleration. Playing with external accelerated GEMM
- [OpenCV China] Zihao Mu: successfully run resnet-50 using the refactored Vulkan backend. The current performance is very low - 170ms on M1 (using Molten VK wrapper), which is ~10x than what Vulkan backend of Tencent NCNN achieves. Now studying the differences
- [OpenCV China] Zheng Wanli & Yuantao Feng: found some regressions in DNN speed in OpenCV 4.7.0 on the palm detection model. The problem seems to be resolved now, PR is submitted.
- made initial version of proposal on how to add ARMv9/SVE2 support into OpenCV.
Shiqi Yu:
- OpenCV 4.7.0 has been tested on Khadas VIM3.
- Tested the latest palm detection model on OpenCV 4.7.0 & Khadas VIM3. The new model works very well.
Stefano Fabri:
- plugin API for OpenXLA is in progress (for integration into PyTorch, JAX, TF). WebAssembly, WebGPU, Vulkan etc. backends will be supported. Possible pipeline is PyTorch => StableHLO => PJRT (IREE) => Vulkan?
Vladimir Dudnik:
- TFLite support is going to be integrated soon.
Alexander S (xperience.ai):
- GSoC multi-camera calibration code is going to be merged soon after Rostislav will double-check the results.
- Done:
  - Rostislav Vasilikhin:
    - Fixed final reprojection error calculation in muti-camera calibration pipeline.
    - Fixed finiteMask (ex. nanMask) PR review remarks from Alexander Alekhin
  - Alexander Smorkalov:
    - Reviewed, tested, helped to resolve issues with #23138, 23237, 23098, opencv-python #794.
    - Investigated ways for GenTL compatible cameras integration to OpenCV video i/o.
  - Alexander Panov:
    - Fixed PR charuco matchImagePoints #23138, added new cpp test
    - Open new issue Structured Append mode support in QR code decoder #23245
    - Testing, fixing, refactoring QR code detect with ArUco algs
    - Fixing pattern layout algorithm
    - Fixing algorithm to pass OpenCV tests (fixed 3 tests, 2 tests left)
    - testing qr codes from issue; Now preparing PR to main OpenCV repo
  - Vladimir Ponomarev:
    - Debugged Video for Windows backend issue. The backend was not fully removed from OpenCV. The final solution will be discussed on the Friday call.
    - Tested and closed PR 22263
- In progress:
  - Alexander Panov: Reviewing PR Include pixel-based uncertainty in ArUco marker detection [#23190] (https://github.com/opencv/opencv/pull/23190)
  - Vladimir Ponomarev: Trackbar layout issue fix (22769).

2023-02-08

Vadim
- [OpenCV China] Feng Yuantao: continue work on optimizing vision transformers
- [OpenCV China] Mu Zihao: the first PR with better Vulkan backend of OpenCV DNN is almost ready, now fixing the failing tests
- [OpenCV China] Zhong Wangli: prepared very efficient palm pose estimation model for OpenCV model zoo, 8-bit quantized model takes 11ms on Intel CPU (model ???) and ~30ms on Khadas VIM3.
- Gave public speech on OpenCV, looks like it was quite popular (100 people attended + a few thousands of online connections)
Shiqi Yu:
- just getting back from long Chinese New Year vacation :)
- played a bit with new low-power intel-based SoC (with Celeron N5105)
Vladimir D (Yadro):
- D. Kurtaev has submitted patch for TFLite support in OpenCV DNN.
- Identified possible co-mentors/consultants for GSoC 2023.
Alexander S (xperience.ai):
- There are some changes in OpenCV team, some members are in, some out
- Done:
  - Rostislav Vasilikhin:
    - nanMask: renamed to finiteMask(), use cases rewritten, tests fixed, SIMD simplified
    - rows & cols bug:
      - the bug reproduced, caused by re-creation during copyTo() call but looks like there are a lot of memory problems yet: valgrind produces up to 1 Mb of logs in unrelated places
    - multicamera calibration:
      - Rebased, tests fixed to build, they fail because of too big reproj error
  - Alexander Smorkalov:
    - Fixed OpenCV-Python build issues on Mac, prepared for bug fix release: #790, 787
    - Made experiments with Wayland support. The activity is frozen till the next major release.
    - Fixed Mac OS warnings after XCode update: #23214, 23203, contrib #3435, 3434.
    - Reviewed, tested, helped to resolve issues with #23098, 23153
  - Alexander Panov:
    - Review ChArUco pre460 pattern support #23153
    - Testing and fixing QR code detect with ArUco algs
    - Fix pattern layout algorithm
    - Update info to google sheets
    - Total detect 78.4%, total decode 59% vs 58.5% detect, 50.5% decode in default algorithm
- In progress:
  - Rostislav Vasilikhin:
    - Multicamera calibration big reprojection error investigation
    - Adding test(s) to marchingCubes()

2023-02-01

Vadim:
- updated list of ideas for GSoC 2023: https://github.com/opencv/opencv/wiki/GSoC_2023. It will be further updated within coming days
- TODO: need to update GSoC 2023 page with the list of successful results from prev GSoC
Gary:
- doing some experimental stuff in an agricultural startup.
Vladimir:
- struggling to run OpenCV on T-head hardware, cannot find toolchain that is compatible with rvv_071 univ intrinsics.
- currently testing OpenCV on the latest compilers (GCC & clang) and QEMU simulator that support RVV 1.0.0.
- consultant's name on RISC-V GSoC project will be provided later.
Stefano:
- please, take a look at https://github.com/openxla/stablehlo/blob/main/docs/spec.md
- also, take a look at http://onnxperience.ai/onnx-mlir/ and https://github.com/onnx/onnx-mlir/issues/1980.
Alexander S (xperience.ai):
- Done:
  - Rostislav Vasilikhin:
    - Stddev calibration: finished, merged, remaining PRs and issues closed
    - Fisheye: fixed in PR #23178, backport to 3.4: PR #23179
    - Omnidir: fixed in PR #3430@contrib for 3.4, merged forward
    - StereoCalib port to 5.x: fixed tests, found rows & cols bug (?)
    - MSVC warnings: fixed in PR #23186
    - rows & cols bug: reproduced, discussion needed
  - Alexander Smorkalov:
    - Tried to enable Wayland support for OpenCV-Python packages on Linux. QT updated, tested with Kubuntu 22.04 with Wayland session. Current solution is not enough for now.
    - Fixed new warnings after compiler update on Mac M1. Submitted PRs to main and contrib repo.
    - Tunned PR testing to skip unstable tests, updated test suppression lists in CI.
    - Reviewed, tested, helped to resolve issues with ## 23168, 23150, 23138 new QR code detector proposal.
  - Alexander Panov:
    - Merged PR remove charuco duplication #3420
    - Fixed PR charuco matchImagePoints #23138, awaiting review
- In progress:
  - Alexander Panov:
    - Fixing PR add python charuco sample #23139
    - Testing and fixing QR code detect with ArUco algs
    - added timing pattern check
    - +10% total decode in best case
    - added info to google sheets
  - Rostislav Vasilikhin:
    - nanMask interface is to be discussed
    - Finishing Egor’s marchingCubes PR
    - Rows & cols bug
  - Alexander Smorkalov:
    - Debugged MacOS 11 dependencies issue on Mac M1. Works on infrastructure solution.
    - New OpenCV-Python release with bug fixes.

2023-01-18

Vadim:
- [OpenCV China]
  - There are some fixes for using OpenMP on macOS.
  - Some further significant acceleration of transformers in OpenCV DNN is expected soon (after Chinese New Year)
  - The work on Vulkan backend of OpenCV DNN is going on
Gary:
- Get ready for GSoC 2023 application: https://summerofcode.withgoogle.com/programs/2023
- Talked with some companies who extended their sponsorship for another year.
- Reviewed submissions for the OpenCV competition.
- Started investigating open NeRF idea.
Shiqi Yu:
- Reviewed all the submissions for OpenCV competition. The winners have been selected and will be announced shortly (likely tomorrow)
Alexander S (xperience.ai):
- Done:
  - Rostislav Vasilikhin:
    - nanMask PR: done & ready for review
    - OCL implementation
    - 1-channel version vectorized
    - cvIsInf(double) bug: found, fixed, backported to 3.4 in PR #23145
    - Tests: parametric test for flags, random Infs and NaNs, reference calculation with SoftFloat
    - stddev calib: fixed 5.x and 3.4 branches
    - USAC fix: updated to remove any code except the fix
  - Alexander Smorkalov:
    - Reviewed, tested, helped to resolve issues: 23141, 22884, 23098, 23145, 23139
    - Debugged OpenCV-Python API issue with zlib. Works on solution and release.
  - Sergei Shutov:
    - Tested all available datasets with OpenCV multicam calibration tool and compared them to Kalibr results (distances + MeshLab renders). In most cases the new OpenCV calibration engine achieves comparable results (hoorray!), but there are few cases where results are not that good. Currently Sergei is preparing proper test cases to reproduce (and then fix) the detected problems.
    - Updated script to load intrinsic parameters from xml files and pairwise distance calculation
    - Fixed a bug with saving results is multiview calibration script (non-homogeneous numpy data)
    - Implemented OpenCV calibration result rendering with meshlab
  - Alexander Panov:
    - Prepared PR add python charuco sample #23139
    - Prepared PR fix charuco matchImagePoints #23138
    - Prepared PR remove charuco duplication #3420
    - Commented issue Charuco marker pose estimation -> Z-axis in wrong direction [#755](https://github.com/opencv/opencv- python/issues/755)
  - In progress:
    - Alexander Panov:
      - preparing QR code detect with ArUco algs. It appears that ArUco algorithms can be applied with some modification for QR code detection and the achieved so-far results are very, very promising.
    - Rostislav Vasilikhin:
      - Porting #22519 to 5.x
      - Extend USAC: analyzing similar existing functionality
    - Sergei Shutov:
      - Compiling calibration and issue reports

2023-01-11

Vadim:
- Paused work on the new version of OpenCV 5 DNN engine.
- Started working on adding the proper 1D and 0D mat support in OpenCV, and also on adding support for other data types (bool, int64/uint64, uint32, bfloat16, ...)
Stefano:
- Currently no connection with OpenCV, just a few updates, especially on PlainML (joint work with Intel):
  - https://github.com/plaidml/tpp-mlir.
  - https://github.com/openxla/community.
  - prepared some nice demo of stable diffusion at CES2023: https://nod.ai/sd-rdna3-ces2023/.
Gary:
- Just got back from vacation. Some negotiations going on; will hopefully have some of the gold OpenCV sponsors to renew/extend the contract.
Shiqi:
- OpenCV Competition: 45 submissions. All the submissions have been reviewed. A few submissions are really brilliant. Have some home work to do before the next competition to provide more detailed instructions how to present the results, which will hopefully increase the number of good submissions.
- Let's plan joint face-to-face OpenCV meeting during 2023 Spring; the date and place to be discussed separately.
Alexander S (xperience.ai):
- OpenCV 4.7.0 has been released. Big thanks to OpenCV team @ Yadro (Alexander Alekhin in particular).
- Rostislav:
  - Stddev calibration issue: found similar issue in omnidir calibration, opened issues [#23057] (https://github.com/opencv/opencv/issues/23057) and #3406@contrib
  - Extend USAC: rewrote Umeyama algorithm so that it’s easy to read, +20% faster and can be potentially vectorized. PR approved although weights are not checked (can be fixed in next PRs)
  - Found a bug in USAC (dense matrix allocation), fixed by #23073
  - Stddev calibration issue: total variable in stddev calculation should be 2x bigger (as it's said in MVG book), the denominator correction is OK (Vadim confirms + lamm45 confirms + MVG explains)
  - Fixing PR’s done, backported & squashed: #22992 to 5.x, #22996 to 4.x, #23025 to 3.4
  - Looks like fisheye calibration has the same problem with stddevs as the usual calibration, see this comment, need to investigate all calibration in OpenCV
    - Fisheye aspect ratio issue:
  - PR fixes aspect ratio calculation, enables Java test, ground truth values & wrong test behavior (rewriting testdata), halves test images to fit into patch size checker
  - Backported: #22604 to 4.x + #1028@extra (merged), #23023 has been replaced by #23029 to 3.4 + #1030@extra (merged), #23027 to 5.x + #1029@extra
- Sergei:
  - Aruco tutorials: fixed tutorial code, text, links and snippets, check it with opencv without contrib, prepared PR in main repo and contrib.
  - Investigate multi camera calibration: build opencv 5.x (PR #22363) and study files in PR; Build doxygen docs and read multiview camera calibration tutorial up to algorithm implementation details.
  - Aruco tutorials: fixed tutorial links and snippets, check it with opencv without contrib, prepared PR in main repo.
  - Reviewed PR #23012 (tried only with cuda 11.8)
  - Update charuco tutorials and sample code in contrib repo and created PR #3407
- Alexander Smorkalov:
  - Released OpenCV and OpenCV Python
  - Updated Python build scripts for FFmpeg 5.x, Python 3.11 support
  - Re-implemented Board class and children for Aruco-based boards in PR #22986.
- Vacations
  - Alexander Panov:

2023

2023 OpenCV Technical Committee

Template

2023-mm-dd

Agenda

Minutes

To Dos

2023-12-20

2023-12-06

2023-11-29

2023-11-22

2023-11-15

2023-11-08

2023-11-01

2023-10-25

2023-10-18

2023-10-11

2023-10-04

2023-09-27

2023-09-20

2023-09-13

2023-09-06

2023-08-30

2023-08-09

2023-08-02

2023-07-26

2023-07-19

2023-06-28

2023-06-21

2023-06-14

2023-06-07

2023-05-31

2023-05-24

2023-05-10

2023-04-26

2023-05-03

2023-04-19

2023-04-12

2023-04-05

2023-03-29

2023-03-22

2023-03-15

2023-03-08

2023-03-01

2023-02-22

2023-02-15

2023-02-08

2023-02-01

2023-01-18

2023-01-11

Clone this wiki locally