G-API OV backend requires cv::MediaFrame #24938

DariaMityagina · 2024-01-29T12:38:43Z

Pull Request Readiness Checklist

Background_subtraction demo G-API issue. Update:

Porting to API20 resulted in an error (both for CPU and NPU):

[ERROR] OpenCV(4.9.0-dev) /home/runner/work/open_model_zoo/open_model_zoo/cache/opencv/modules/gapi/src/backends/ov/govbackend.cpp:813: error: (-215: assertion not done ) cv::util::holds_alternative<cv::GMatDesc>(input_meta) in function 'cfgPreProcessing'

Adding cv::MediaFrame support to govbackend resulted in the following (tested with CPU):

TODO

As part of the review process this comment was addressed which make it impossible to run the demo. I will bring those changes back in a separate PR [support PartialShape]

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

I agree to contribute to the project under Apache 2 License.
To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
The PR is proposed to the proper branch
There is a reference to the original bug report and related work
There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
The feature is well documented and sample code can be built with the project CMake

DariaMityagina · 2024-01-30T12:56:44Z

Need to investigate this issue in one of the test cases - WIP:
https://github.com/openvinotoolkit/open_model_zoo/actions/runs/7700178002/job/20983409804?pr=3903

[ ERROR ] OpenCV(4.9.0-dev) /home/runner/work/open_model_zoo/open_model_zoo/cache/opencv-with-fix/opencv/modules/core/src/matrix_wrap.cpp:1667: error: (-215:Assertion failed) !fixedSize() in function 'release'

TolyaTalamanov · 2024-01-30T15:51:12Z

Need to investigate this issue in one of the test cases - WIP: https://github.com/openvinotoolkit/open_model_zoo/actions/runs/7700178002/job/20983409804?pr=3903
[ ERROR ] OpenCV(4.9.0-dev) /home/runner/work/open_model_zoo/open_model_zoo/cache/opencv-with-fix/opencv/modules/core/src/matrix_wrap.cpp:1667: error: (-215:Assertion failed) !fixedSize() in function 'release'

Will be glad to help if it hasn't been resolved yet

DariaMityagina · 2024-01-31T12:52:31Z

Need to investigate this issue in one of the test cases - WIP: https://github.com/openvinotoolkit/open_model_zoo/actions/runs/7700178002/job/20983409804?pr=3903
[ ERROR ] OpenCV(4.9.0-dev) /home/runner/work/open_model_zoo/open_model_zoo/cache/opencv-with-fix/opencv/modules/core/src/matrix_wrap.cpp:1667: error: (-215:Assertion failed) !fixedSize() in function 'release'
Will be glad to help if it hasn't been resolved yet

Thanks a lot!
I think I found the problem, I'm working on a fix now.

DariaMityagina · 2024-02-02T10:39:24Z

Opened this PR for review.

This issue is going to be addressed in a follow-up ticket - 111291.
Since it only affected instance-segmentation-security-0091 case in OMZ demos check, I only enabled passing instance-segmentation-person-0007 case for now (please see openvinotoolkit/open_model_zoo#3903).

DariaMityagina · 2024-02-02T10:42:11Z

@TolyaTalamanov could you please review this PR?

DariaMityagina · 2024-02-02T21:08:04Z

modules/gapi/test/infer/gapi_infer_ov_tests.cpp

+    // Assert
+    validate();
+}
+


opencv_test_gapi --gtest_filter=*MediaFrameTestAgeGenderOV* CTEST_FULL_OUTPUT OpenCV version: 4.9.0-dev OpenCV VCS version: 4.9.0-106-gbde2400e29-dirty Build type: Release Compiler: /usr/bin/c++ (ver 9.4.0) Parallel framework: pthreads (nthreads=6) CPU features: SSE SSE2 SSE3 *SSE4.1 *SSE4.2 *FP16 *AVX *AVX2 *AVX512-SKX? Intel(R) IPP version: ippIP AVX2 (l9) 2021.10.0 (-) Sep 18 2023 Intel(R) IPP features code: 0x8000 OpenCL is disabled TEST: Skip tests with tags: 'mem_6gb', 'verylong' Note: Google Test filter = *MediaFrameTestAgeGenderOV* [==========] Running 2 tests from 1 test case. [----------] Global test environment set-up. [----------] 2 tests from MediaFrameTestAgeGenderOV [ RUN ] MediaFrameTestAgeGenderOV.InferMediaInputBGR [ OK ] MediaFrameTestAgeGenderOV.InferMediaInputBGR (118 ms) [ RUN ] MediaFrameTestAgeGenderOV.InferROIGenericMediaInputBGR [ OK ] MediaFrameTestAgeGenderOV.InferROIGenericMediaInputBGR (72 ms) [----------] 2 tests from MediaFrameTestAgeGenderOV (190 ms total) [----------] Global test environment tear-down [==========] 2 tests from 1 test case ran. (190 ms total) [ PASSED ] 2 tests.

Cool! Can we also cover NV12 test cases?

Done, thanks! 24e5ceb

DariaMityagina · 2024-02-12T07:27:20Z

@TolyaTalamanov could you please review this PR?

modules/gapi/src/backends/ov/govbackend.cpp

DariaMityagina · 2024-02-15T11:46:19Z

@asmorkalov hi! Could you please advise how to run all pre-commit checks, not just the default check?

asmorkalov · 2024-02-15T12:21:26Z

@DariaMityagina Github actions are triggered automatically for OpenCV organization members only. I approved built for you.

DariaMityagina · 2024-02-15T12:26:02Z

@DariaMityagina Github actions are triggered automatically for OpenCV organization members only. I approved built for you.

Great, thank you!

modules/gapi/src/backends/ov/govbackend.cpp

TolyaTalamanov · 2024-02-15T14:09:28Z

modules/gapi/test/infer/gapi_infer_ov_tests.cpp

+    // Assert
+    validate();
+}
+


Cool! Can we also cover NV12 test cases?

DariaMityagina · 2024-04-01T06:08:01Z

@DariaMityagina @TolyaTalamanov Friendly reminder.

Hello! Will get back to this PR soon. Thanks!

…x code style

DariaMityagina · 2024-04-15T17:01:56Z

@DariaMityagina Github actions are triggered automatically for OpenCV organization members only. I approved built for you.

@asmorkalov hi! Could you please trigger the checks again?

DariaMityagina · 2024-04-16T09:08:51Z

@TolyaTalamanov hi! Could you please take a look at the PR?

DariaMityagina · 2024-04-16T12:19:36Z

@asmorkalov hi! Could you please trigger the checks again? I fixed tests failure.

DariaMityagina · 2024-04-19T11:30:10Z

@asmorkalov hi! Could you please trigger the checks again?

modules/gapi/src/backends/ov/govbackend.cpp

TolyaTalamanov

Great, thank you so much 🔥

LGTM 👍

…ackend

DariaMityagina · 2024-04-23T15:15:41Z

@asmorkalov hi! Can this PR be merged? Could you please take a look?

dmatveev

Fantastic work, thanks @DariaMityagina !

DariaMityagina · 2024-04-24T13:29:34Z

@asmorkalov thanks!

Fixed build warnings introduced in #24938

…frame-support-to-govbackend G-API OV backend requires cv::MediaFrame opencv#24938 ### Pull Request Readiness Checklist **Background_subtraction demo G-API issue. Update:** Porting to API20 resulted in an error (both for CPU and NPU): ``` [ERROR] OpenCV(4.9.0-dev) /home/runner/work/open_model_zoo/open_model_zoo/cache/opencv/modules/gapi/src/backends/ov/govbackend.cpp:813: error: (-215: assertion not done ) cv::util::holds_alternative<cv::GMatDesc>(input_meta) in function 'cfgPreProcessing' ``` Adding cv::MediaFrame support to govbackend resulted in the following (tested with CPU): <img width="941" alt="image" src="https://github.com/opencv/opencv/assets/52502732/3a003d61-bda7-4b1e-9117-3410cda1ba32"> ### TODO - [ ] **As part of the review process [this comment](opencv#24938 (comment)) was addressed which make it impossible to run the demo. I will bring those changes back in a separate PR [support `PartialShape`]** See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request - [x] I agree to contribute to the project under Apache 2 License. - [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV - [x] The PR is proposed to the proper branch - [x] There is a reference to the original bug report and related work - [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable Patch to opencv_extra has the same branch name. - [ ] The feature is well documented and sample code can be built with the project CMake

DariaMityagina mentioned this pull request Jan 29, 2024

Port G-API demos to API2.0 - background_subtraction_demo/cpp_gapi openvinotoolkit/open_model_zoo#3903

Draft

2 tasks

TolyaTalamanov self-assigned this Jan 30, 2024

TolyaTalamanov self-requested a review January 30, 2024 12:48

TolyaTalamanov added feature category: t-api cv::UMat related issues labels Jan 30, 2024

DariaMityagina force-pushed the icv/dm/add-media-frame-support-to-govbackend branch 3 times, most recently from 8e87508 to 633ccf9 Compare February 2, 2024 10:24

DariaMityagina marked this pull request as ready for review February 2, 2024 10:35

DariaMityagina changed the title ~~G-API OV backend requires cv::MediaFrame~~ G-API OV backend requires cv::MediaFrame - part 1 Feb 2, 2024

DariaMityagina force-pushed the icv/dm/add-media-frame-support-to-govbackend branch 3 times, most recently from 34960ec to 58cb0ff Compare February 2, 2024 21:07

DariaMityagina commented Feb 2, 2024

View reviewed changes

DariaMityagina changed the title ~~G-API OV backend requires cv::MediaFrame - part 1~~ G-API OV backend requires cv::MediaFrame Feb 12, 2024

asmorkalov added this to the 4.10.0 milestone Feb 12, 2024

DariaMityagina force-pushed the icv/dm/add-media-frame-support-to-govbackend branch 2 times, most recently from e58497b to 17cbb67 Compare February 13, 2024 10:17

TolyaTalamanov requested changes Feb 13, 2024

View reviewed changes

DariaMityagina requested a review from TolyaTalamanov February 14, 2024 12:07

TolyaTalamanov reviewed Feb 15, 2024

View reviewed changes

DariaMityagina added 4 commits April 15, 2024 08:40

Initial changes to support MediaFrame

8248a39

Initial changes to support MediaFrame - review

7d30147

Initial changes to support MediaFrame - review - remove cfgColorFormat

0f5220b

Initial changes to support MediaFrame - review - refactor and add tests

38c0ee2

DariaMityagina force-pushed the icv/dm/add-media-frame-support-to-govbackend branch from 833bd89 to b1a8203 Compare April 15, 2024 14:53

Initial changes to support MediaFrame - review - refactor/clean up/fi…

7d58666

…x code style

DariaMityagina force-pushed the icv/dm/add-media-frame-support-to-govbackend branch from b1a8203 to 7d58666 Compare April 15, 2024 16:53

DariaMityagina requested a review from TolyaTalamanov April 15, 2024 16:55

Initial changes to support MediaFrame - review - test image

655777a

TolyaTalamanov reviewed Apr 22, 2024

View reviewed changes

Initial changes to support MediaFrame - review - p2

2f249a8

DariaMityagina requested a review from TolyaTalamanov April 23, 2024 12:27

TolyaTalamanov approved these changes Apr 23, 2024

View reviewed changes

Merge branch 'opencv:4.x' into icv/dm/add-media-frame-support-to-govb…

584ace9

…ackend

dmatveev approved these changes Apr 24, 2024

View reviewed changes

asmorkalov merged commit ebea657 into opencv:4.x Apr 24, 2024
27 of 28 checks passed

asmorkalov added a commit to asmorkalov/opencv that referenced this pull request Apr 24, 2024

Fixed build warnings introduced in opencv#24938

2507ac9

asmorkalov mentioned this pull request Apr 24, 2024

Fixed build warnings introduced in #24938 #25486

Merged

6 tasks

asmorkalov added a commit to asmorkalov/opencv that referenced this pull request Apr 25, 2024

Fixed build warnings introduced in opencv#24938

7bad12c

asmorkalov added a commit that referenced this pull request Apr 25, 2024

Merge pull request #25486 from asmorkalov:as/gapi_warning_fix

7b58ffe

Fixed build warnings introduced in #24938

DariaMityagina mentioned this pull request Apr 29, 2024

G-API OV backend requires partial shape support #25508

Draft

7 tasks

klatism pushed a commit to klatism/opencv that referenced this pull request May 17, 2024

Fixed build warnings introduced in opencv#24938

d5f18eb

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

G-API OV backend requires cv::MediaFrame #24938

G-API OV backend requires cv::MediaFrame #24938

DariaMityagina commented Jan 29, 2024 •

edited

DariaMityagina commented Jan 30, 2024

TolyaTalamanov commented Jan 30, 2024 •

edited

DariaMityagina commented Jan 31, 2024 •

edited

DariaMityagina commented Feb 2, 2024

DariaMityagina commented Feb 2, 2024

DariaMityagina Feb 2, 2024

TolyaTalamanov Feb 15, 2024

DariaMityagina Feb 19, 2024

DariaMityagina commented Feb 12, 2024

DariaMityagina commented Feb 15, 2024

asmorkalov commented Feb 15, 2024

DariaMityagina commented Feb 15, 2024

TolyaTalamanov Feb 15, 2024

DariaMityagina commented Apr 1, 2024

DariaMityagina commented Apr 15, 2024

DariaMityagina commented Apr 16, 2024

DariaMityagina commented Apr 16, 2024

DariaMityagina commented Apr 19, 2024

TolyaTalamanov left a comment

DariaMityagina commented Apr 23, 2024 •

edited

dmatveev left a comment

DariaMityagina commented Apr 24, 2024

G-API OV backend requires cv::MediaFrame #24938

G-API OV backend requires cv::MediaFrame #24938

Conversation

DariaMityagina commented Jan 29, 2024 • edited

Pull Request Readiness Checklist

TODO

DariaMityagina commented Jan 30, 2024

TolyaTalamanov commented Jan 30, 2024 • edited

DariaMityagina commented Jan 31, 2024 • edited

DariaMityagina commented Feb 2, 2024

DariaMityagina commented Feb 2, 2024

DariaMityagina Feb 2, 2024

Choose a reason for hiding this comment

TolyaTalamanov Feb 15, 2024

Choose a reason for hiding this comment

DariaMityagina Feb 19, 2024

Choose a reason for hiding this comment

DariaMityagina commented Feb 12, 2024

DariaMityagina commented Feb 15, 2024

asmorkalov commented Feb 15, 2024

DariaMityagina commented Feb 15, 2024

TolyaTalamanov Feb 15, 2024

Choose a reason for hiding this comment

DariaMityagina commented Apr 1, 2024

DariaMityagina commented Apr 15, 2024

DariaMityagina commented Apr 16, 2024

DariaMityagina commented Apr 16, 2024

DariaMityagina commented Apr 19, 2024

TolyaTalamanov left a comment

Choose a reason for hiding this comment

DariaMityagina commented Apr 23, 2024 • edited

dmatveev left a comment

Choose a reason for hiding this comment

DariaMityagina commented Apr 24, 2024

DariaMityagina commented Jan 29, 2024 •

edited

TolyaTalamanov commented Jan 30, 2024 •

edited

DariaMityagina commented Jan 31, 2024 •

edited

DariaMityagina commented Apr 23, 2024 •

edited