Fix gstreamer backend with manual pipelines #24243

kecsap · 2023-09-07T18:35:52Z

Fix broken seeking in audio/video playback
Fix broken audio playback
Fix unreliable seeking
Estimate frame count if it is not available directly
Return -1 for frame count and fps if it is not available.
Return 0 for fps if the video has variable frame rate
Enable and fix tests

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

I agree to contribute to the project under Apache 2 License.
To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
The PR is proposed to the proper branch
[-] There is a reference to the original bug report and related work => Reproducible test provided
[-] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
The feature is well documented and sample code can be built with the project CMake

Download two test videos:

wget https://github.com/ietf-wg-cellar/matroska-test-files/raw/master/test_files/test1.mkv
wget https://test-videos.co.uk/vids/jellyfish/mkv/360/Jellyfish_360_10s_5MB.mkv

I modified a OpenCV videoio sample to demonstrate the problem, here it is the patch: http://dpaste.com//C9MAT2K6W
Build the sample, on Ubuntu:

g++ -g videocapture_audio_combination.cpp -I/usr/include/opencv4 `pkg-config --libs --cflags opencv4` -o videocapture_audio_combination

Play an audio stream with seeking BEFORE the fix:

$ ./videocapture_audio_combination --audio "filesrc location=test1.mkv ! queue ! matroskademux name=demux demux.audio_0 ! decodebin ! audioconvert ! appsink"[ERROR:0@0.009] global cap.cpp:164 open VIDEOIO(GSTREAMER): raised OpenCV exception:

OpenCV(4.8.0-dev) ./modules/videoio/src/cap_gstreamer.cpp:153: error: (-215:Assertion failed) ptr in function 'get'


[ WARN:0@0.009] global cap.cpp:204 open VIDEOIO(GSTREAMER): backend is generally available but can't be used to capture by name
ERROR! Can't to open file: filesrc location=test1.mkv ! queue ! matroskademux name=demux demux.audio_0 ! decodebin ! audioconvert ! appsink

Play a video stream with seeking BEFORE the fix:

$ ./videocapture_audio_combination --audio "filesrc location=Jellyfish_360_10s_5MB.mkv ! queue ! matroskademux name=demux demux.video_0 ! decodebin ! videoconvert ! video/x-raw, format=BGR ! appsink drop=1"
[ WARN:0@0.034] global cap_gstreamer.cpp:1728 open OpenCV | GStreamer warning: Cannot query video position: status=1, value=22, duration=300
CAP_PROP_AUDIO_DATA_DEPTH: CV_16S
CAP_PROP_AUDIO_SAMPLES_PER_SECOND: 44100
CAP_PROP_AUDIO_TOTAL_CHANNELS: 0
CAP_PROP_AUDIO_TOTAL_STREAMS: [ WARN:0@0.034] global cap_gstreamer.cpp:1898 getProperty OpenCV | GStreamer: CAP_PROP_AUDIO_TOTAL_STREAMS property is not supported
0
[ WARN:0@0.034] global cap_gstreamer.cpp:1817 getProperty OpenCV | GStreamer: CAP_PROP_POS_MSEC property result may be unrealiable: https://github.com/opencv/opencv/issues/19025
Timestamp: 0.6218
Timestamp: 33.1085
Timestamp: 67.1274
Timestamp: 100.1182
Timestamp: 133.1204
Timestamp: 167.1195
Timestamp: 200.1161
Timestamp: 233.1147
Timestamp: 267.1194
Timestamp: 300.1202
[ WARN:0@0.338] global cap_gstreamer.cpp:1949 setProperty OpenCV | GStreamer warning: GStreamer: unable to seek
0:00:00.338215907 3892572 0x5592899c7580 WARN                 basesrc gstbasesrc.c:3127:gst_base_src_loop:<filesrc0> error: Internal data stream error.
0:00:00.338235884 3892572 0x5592899c7580 WARN                 basesrc gstbasesrc.c:3127:gst_base_src_loop:<filesrc0> error: streaming stopped, reason not-linked (-1)
0:00:00.338264287 3892572 0x5592899c7580 WARN                   queue gstqueue.c:992:gst_queue_handle_sink_event:<queue0> error: Internal data stream error.
0:00:00.338270329 3892572 0x5592899c7580 WARN                   queue gstqueue.c:992:gst_queue_handle_sink_event:<queue0> error: streaming stopped, reason not-linked (-1)
[ WARN:0@0.339] global cap_gstreamer.cpp:2784 handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module filesrc0 reported: Internal data stream error.
[ WARN:0@0.339] global cap_gstreamer.cpp:1199 startPipeline OpenCV | GStreamer warning: unable to start pipeline
Number of audio samples: 0
Number of video frames: 10
[ WARN:0@0.339] global cap_gstreamer.cpp:1164 isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created

Play an audio stream with seeking AFTER the fix:

$ ./videocapture_audio_combination --audio "filesrc location=test1.mkv ! queue ! matroskademux name=demux demux.audio_0 ! decodebin ! audioconvert ! appsink"CAP_PROP_AUDIO_DATA_DEPTH: CV_16S
CAP_PROP_AUDIO_SAMPLES_PER_SECOND: 48000
CAP_PROP_AUDIO_TOTAL_CHANNELS: 2
CAP_PROP_AUDIO_TOTAL_STREAMS: [ WARN:0@0.025] global cap_gstreamer.cpp:1903 getProperty OpenCV | GStreamer: CAP_PROP_AUDIO_TOTAL_STREAMS property is not supported
0
Timestamp: 0.0000
Timestamp: 24.0000
Timestamp: 48.0000
Timestamp: 72.0000
Timestamp: 96.0000
Timestamp: 120.0000
Timestamp: 144.0000
Timestamp: 168.0000
Timestamp: 192.0000
Timestamp: 216.0000
Timestamp: 3500.0000
Timestamp: 3504.0000
Timestamp: 3528.0000
Timestamp: 3552.0000
Timestamp: 3576.0000
Timestamp: 3600.0000
Timestamp: 3624.0000
Timestamp: 3648.0000
Timestamp: 3672.0000
Timestamp: 3696.0000
Timestamp: 3720.0000
Timestamp: 3744.0000
Timestamp: 3768.0000
Timestamp: 3792.0000
Timestamp: 3816.0000
Timestamp: 3840.0000
Timestamp: 3864.0000
Timestamp: 3888.0000
Timestamp: 3912.0000
Timestamp: 3936.0000

Play a video stream with seeking AFTER the fix:

$ ./videocapture_audio_combination --audio "filesrc location=Jellyfish_360_10s_5MB.mkv ! queue ! matroskademux name=demux demux.video_0 ! decodebin ! videoconvert ! video/x-raw, format=BGR ! appsink drop=1"
[ WARN:0@0.033] global cap_gstreamer.cpp:1746 open OpenCV | GStreamer warning: Cannot query video position: status=1, value=22, duration=300
CAP_PROP_AUDIO_DATA_DEPTH: CV_16S
CAP_PROP_AUDIO_SAMPLES_PER_SECOND: 44100
CAP_PROP_AUDIO_TOTAL_CHANNELS: 0
CAP_PROP_AUDIO_TOTAL_STREAMS: [ WARN:0@0.034] global cap_gstreamer.cpp:1903 getProperty OpenCV | GStreamer: CAP_PROP_AUDIO_TOTAL_STREAMS property is not supported
0
Timestamp: 0.0000
Timestamp: 33.0000
Timestamp: 67.0000
Timestamp: 100.0000
Timestamp: 133.0000
Timestamp: 167.0000
Timestamp: 200.0000
Timestamp: 233.0000
Timestamp: 267.0000
Timestamp: 300.0000
0:00:00.335931693 3893501 0x55bbe76ad920 WARN      matroskareadcommon matroska-read-common.c:759:gst_matroska_read_common_parse_skip:<demux:sink> Unknown CueTrackPositions subelement 0xf0 - ignoring
0:00:00.335952823 3893501 0x55bbe76ad920 WARN      matroskareadcommon matroska-read-common.c:759:gst_matroska_read_common_parse_skip:<demux:sink> Unknown CueTrackPositions subelement 0xf0 - ignoring
0:00:00.335988029 3893501 0x55bbe76ad920 WARN                 basesrc gstbasesrc.c:1742:gst_base_src_perform_seek:<filesrc0> duplicate event found 184
Timestamp: 3467.0000
Timestamp: 3500.0000
Timestamp: 3533.0000
Timestamp: 3567.0000
Timestamp: 3600.0000
Timestamp: 3633.0000
Timestamp: 3667.0000
Timestamp: 3700.0000
Timestamp: 3733.0000
Timestamp: 3767.0000
Timestamp: 3800.0000
Timestamp: 3833.0000
Timestamp: 3867.0000
Timestamp: 3900.0000
Timestamp: 3933.0000
Timestamp: 3967.0000
Timestamp: 4000.0000
Timestamp: 4033.0000
Timestamp: 4067.0000
Timestamp: 4100.0000

force_builders=linux,Linux OpenCL,Linux AVX2,docs

asmorkalov · 2023-09-08T08:35:08Z

@kecsap Thanks a lot for your contribution. The GSTreamer issue is known and the related test case is disable for now. Could you enable it and check on your side: https://github.com/opencv/opencv/blob/757d2963823efd00f92041a59bd26d18c2e76728/modules/videoio/test/test_video_io.cpp#L203C18-L203C18

kecsap · 2023-09-13T09:07:38Z

@kecsap Thanks a lot for your contribution. The GSTreamer issue is known and the related test case is disable for now. Could you enable it and check on your side: https://github.com/opencv/opencv/blob/757d2963823efd00f92041a59bd26d18c2e76728/modules/videoio/test/test_video_io.cpp#L203C18-L203C18

I enabled the tests and fixed them. Along with these actions, a new video duration estimation is implemented if the exact frame count is unavailable. I also added a "queue" element to the default pipeline because it improves the pipeline behavior for certain formats. Although I have to say some test videos do not seem to me very standard, different video tools on Ubuntu Linux report different numbers and metrics.

modules/videoio/test/test_video_io.cpp

asmorkalov · 2023-09-13T13:18:43Z

@kecsap Thanks for the contribution. Looks like the behaviour hardly depends on GStreamer verison and installed plugins set. Some issues With Ubuntu 18.04:

[ RUN      ] videoio/videoio_bunny.read_position/6, where GetParam() = ("mp4", GSTREAMER)
[ WARN:0@0.198] global cap_gstreamer.cpp:1740 open OpenCV | GStreamer warning: frame count is estimated by duration and fps
/home/alexander/Projects/OpenCV/opencv-master/modules/videoio/test/test_video_io.cpp:40: Failure
Expected equality of these values:
  idx
    Which is: 20
  (int)cap.get(CAP_PROP_POS_FRAMES)
    Which is: 10
Google Test trace:
/home/alexander/Projects/OpenCV/opencv-master/modules/videoio/test/test_video_io.cpp:97: progressive seek
/home/alexander/Projects/OpenCV/opencv-master/modules/videoio/test/test_video_io.cpp:40: Failure
Expected equality of these values:
  idx
    Which is: 5
  (int)cap.get(CAP_PROP_POS_FRAMES)
    Which is: 0
Google Test trace:
/home/alexander/Projects/OpenCV/opencv-master/modules/videoio/test/test_video_io.cpp:111: random seek
[  FAILED  ] videoio/videoio_bunny.rea_position/6, where GetParam() = ("mp4", GSTREAMER) (199 ms)
...
[ RUN      ] videoio/videoio_bunny.read_position/0, where GetParam() = ("wmv", GSTREAMER)
[ WARN:0@0.217] global cap_gstreamer.cpp:1740 open OpenCV | GStreamer warning: frame count is estimated by duration and fps
/home/alexander/Projects/OpenCV/opencv-master/modules/videoio/test/test_video_io.cpp:28: Failure
Value of: img.empty()
  Actual: true
Expected: false
idx=125
Google Test trace:
/home/alexander/Projects/OpenCV/opencv-master/modules/videoio/test/test_video_io.cpp:72: consecutive read
/home/alexander/Projects/OpenCV/opencv-master/modules/videoio/test/test_video_io.cpp:28: Failure
Value of: img.empty()
  Actual: true
Expected: false
idx=126
Google Test trace:
/home/alexander/Projects/OpenCV/opencv-master/modules/videoio/test/test_video_io.cpp:72: consecutive read
/home/alexander/Projects/OpenCV/opencv-master/modules/videoio/test/test_video_io.cpp:28: Failure
Value of: img.empty()
  Actual: true
Expected: false
idx=127
Google Test trace:
/home/alexander/Projects/OpenCV/opencv-master/modules/videoio/test/test_video_io.cpp:72: consecutive read
/home/alexander/Projects/OpenCV/opencv-master/modules/videoio/test/test_video_io.cpp:28: Failure
Value of: img.empty()
  Actual: true
Expected: false
idx=128
Google Test trace:
/home/alexander/Projects/OpenCV/opencv-master/modules/videoio/test/test_video_io.cpp:72: consecutive read
...
[ RUN      ] videoio/videoio_bunny.frame_count/0, where GetParam() = ("wmv", GSTREAMER)
[ WARN:0@0.184] global cap_gstreamer.cpp:1740 open OpenCV | GStreamer warning: frame count is estimated by duration and fps
/home/alexander/Projects/OpenCV/opencv-master/modules/videoio/test/test_video_io.cpp:173: Failure
Expected equality of these values:
  bunny_param.getCount()
    Which is: 125
  count_prop
    Which is: 130
/home/alexander/Projects/OpenCV/opencv-master/modules/videoio/test/test_video_io.cpp:192: Failure
The difference between bunny_param.getCount() and count_actual is 115, which exceeds 1, where
bunny_param.getCount() evaluates to 125,
count_actual evaluates to 10, and
1 evaluates to 1.
[  FAILED  ] videoio/videoio_bunny.frame_count/0, where GetParam() = ("wmv", GSTREAMER) (65 ms)

kecsap · 2023-09-13T13:39:08Z

@kecsap Thanks for the contribution. Looks like the behaviour hardly depends on GStreamer verison and installed plugins set. Some issues With Ubuntu 18.04:

Do not these tests fail before my changes? I did not touch CAP_PROP_FRAME_COUNT, only CAP_PROP_POS_MSEC in this MR.

asmorkalov · 2023-09-13T13:51:39Z

Ubuntu 22.04 produce the same failures + Ubuntu 22.04 issues. There is mjpeg regression there: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/1122

kecsap · 2023-09-20T20:48:23Z

Ubuntu 22.04 produce the same failures + Ubuntu 22.04 issues. There is mjpeg regression there: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/1122

Now the mentioned tests pass.

kecsap · 2023-09-26T07:57:51Z

@asmorkalov Pinging... Are the changes okay now?

modules/videoio/src/cap_gstreamer.cpp

kecsap · 2023-10-09T20:48:28Z

I have addressed the comments and rebased the MR on the top of the branch 4.x.

kecsap · 2023-10-17T07:43:45Z

@mshabunin Ping-pong. Please review if you have time. :)

asmorkalov

👍

kecsap · 2023-10-19T11:26:16Z

Are there any other things to do on my side before you merge? Waiting for second approval? Rebasing?

modules/videoio/src/cap_gstreamer.cpp

mshabunin · 2023-10-20T14:50:38Z

BTW, I have this example app which works similarly to OpenCV GStreamer capture (https://github.com/mshabunin/gstreamer-example) and it demonstrates that GStreamer might not be reliable when evaluating stream position and timestamps.

E.g. gst_element_query_position can return 1300 ms before first sample being retrieved, or first BUFFER_PTS can be 200 ms for mp4 file and 0 ms for mkv file.

mshabunin

It would be nice to have GStreamer-specific seeking test, because we don't have one (see test_gstreamer.cpp). It can use any video file from testdata (preferably the one which is the most reliable) or generate temporary synthetic file and also check both manual and automatic pipeline construction.

modules/videoio/src/cap_gstreamer.cpp

kecsap · 2023-10-27T11:17:21Z

"GStreamer might not be reliable when evaluating stream position and timestamps."

I politely would like to express, it is not my business, I am not a GStreamer developer. I made changes based on timestamps extracted from the acquired buffers which is much more reliable than the earlier implementation in this backend and it does not crash as before in some cases. It is far from perfect, but much better than it was before.

"It would be nice to have GStreamer-specific seeking test, because we don't have one (see test_gstreamer.cpp)."

Is not it already there in modules/videoio/test/test_video_io.cpp?

I rebased the changes based on the latest 4.x branch.
I removed some failing GStreamer tests for MJPEG codec that does not work at all.

mshabunin

I politely would like to express, it is not my business, I am not a GStreamer developer. I made changes based on timestamps extracted from the acquired buffers which is much more reliable than the earlier implementation in this backend and it does not crash as before in some cases. It is far from perfect, but much better than it was before.

"It would be nice to have GStreamer-specific seeking test, because we don't have one (see test_gstreamer.cpp)."

Is not it already there in modules/videoio/test/test_video_io.cpp?

Apparently some scenarios are covered. You've fixed some scenario (I modified a OpenCV videoio sample to demonstrate the problem), but the paste you shared is not available and it is not covered by our tests. Is it possible to add it to the tests to be sure future modifications wouldn't break it?

Note:
I've prepared a framework for OpenCV-GStreamer testing (https://github.com/mshabunin/gstreamer-check). It helped to determine following things:

for some reason GStreamer audio tests hang or crash or fail in both cases: with patch and without patch
MJPG-related GStreamer tests fail only in Ubuntu 22, in more recent versions of GStreamer (Ubuntu 23, Fedora, manually-built latest GStreamer), these tests pass

This behavior can be related to the system configuration or installed plugins or HW availability in container, because on my local system all tests pass with manually-built latest GStreamer release.

modules/videoio/src/cap_gstreamer.cpp

- Fix broken seeking in audio/video playback - Fix broken audio playback - Fix unreliable seeking (read out from the last acquired buffer) - Estimate frame count if it is not available directly - Return -1 for frame count and fps if it is not available. - Return 0 for fps if the video has variable frame rate - Enable and fix tests

kecsap · 2023-11-02T20:57:00Z

@mshabunin

Okay, a new round of updates are up:

I fixed your comments, but you can reopen if you see a resolution is wrong.
I enabled back the MJPG test cases.
I complemented a test with a manual GStreamer pipeline seeking case in test_video_io.cpp.

mshabunin

I've moved new test to test_gstreamer.cpp and tested it on various platforms.

Fixed build warning introduced in #24243

Fix gstreamer backend with manual pipelines opencv#24243 - Fix broken seeking in audio/video playback - Fix broken audio playback - Fix unreliable seeking - Estimate frame count if it is not available directly - Return -1 for frame count and fps if it is not available. - Return 0 for fps if the video has variable frame rate - Enable and fix tests ### Pull Request Readiness Checklist See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request - [X] I agree to contribute to the project under Apache 2 License. - [X] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV - [X] The PR is proposed to the proper branch - [-] There is a reference to the original bug report and related work => Reproducible test provided - [-] There is accuracy test, performance test and test data in opencv_extra repository, if applicable Patch to opencv_extra has the same branch name. - [X] The feature is well documented and sample code can be built with the project CMake 1. Download two test videos: ```bash wget https://github.com/ietf-wg-cellar/matroska-test-files/raw/master/test_files/test1.mkv wget https://test-videos.co.uk/vids/jellyfish/mkv/360/Jellyfish_360_10s_5MB.mkv ``` 2. I modified a OpenCV videoio sample to demonstrate the problem, here it is the patch: http://dpaste.com//C9MAT2K6W 3. Build the sample, on Ubuntu: ```bash g++ -g videocapture_audio_combination.cpp -I/usr/include/opencv4 `pkg-config --libs --cflags opencv4` -o videocapture_audio_combination ``` 4. Play an audio stream with seeking BEFORE the fix: ```bash $ ./videocapture_audio_combination --audio "filesrc location=test1.mkv ! queue ! matroskademux name=demux demux.audio_0 ! decodebin ! audioconvert ! appsink"[ERROR:0@0.009] global cap.cpp:164 open VIDEOIO(GSTREAMER): raised OpenCV exception: OpenCV(4.8.0-dev) ./modules/videoio/src/cap_gstreamer.cpp:153: error: (-215:Assertion failed) ptr in function 'get' [ WARN:0@0.009] global cap.cpp:204 open VIDEOIO(GSTREAMER): backend is generally available but can't be used to capture by name ERROR! Can't to open file: filesrc location=test1.mkv ! queue ! matroskademux name=demux demux.audio_0 ! decodebin ! audioconvert ! appsink ``` 5. Play a video stream with seeking BEFORE the fix: ```bash $ ./videocapture_audio_combination --audio "filesrc location=Jellyfish_360_10s_5MB.mkv ! queue ! matroskademux name=demux demux.video_0 ! decodebin ! videoconvert ! video/x-raw, format=BGR ! appsink drop=1" [ WARN:0@0.034] global cap_gstreamer.cpp:1728 open OpenCV | GStreamer warning: Cannot query video position: status=1, value=22, duration=300 CAP_PROP_AUDIO_DATA_DEPTH: CV_16S CAP_PROP_AUDIO_SAMPLES_PER_SECOND: 44100 CAP_PROP_AUDIO_TOTAL_CHANNELS: 0 CAP_PROP_AUDIO_TOTAL_STREAMS: [ WARN:0@0.034] global cap_gstreamer.cpp:1898 getProperty OpenCV | GStreamer: CAP_PROP_AUDIO_TOTAL_STREAMS property is not supported 0 [ WARN:0@0.034] global cap_gstreamer.cpp:1817 getProperty OpenCV | GStreamer: CAP_PROP_POS_MSEC property result may be unrealiable: opencv#19025 Timestamp: 0.6218 Timestamp: 33.1085 Timestamp: 67.1274 Timestamp: 100.1182 Timestamp: 133.1204 Timestamp: 167.1195 Timestamp: 200.1161 Timestamp: 233.1147 Timestamp: 267.1194 Timestamp: 300.1202 [ WARN:0@0.338] global cap_gstreamer.cpp:1949 setProperty OpenCV | GStreamer warning: GStreamer: unable to seek 0:00:00.338215907 3892572 0x5592899c7580 WARN basesrc gstbasesrc.c:3127:gst_base_src_loop:<filesrc0> error: Internal data stream error. 0:00:00.338235884 3892572 0x5592899c7580 WARN basesrc gstbasesrc.c:3127:gst_base_src_loop:<filesrc0> error: streaming stopped, reason not-linked (-1) 0:00:00.338264287 3892572 0x5592899c7580 WARN queue gstqueue.c:992:gst_queue_handle_sink_event:<queue0> error: Internal data stream error. 0:00:00.338270329 3892572 0x5592899c7580 WARN queue gstqueue.c:992:gst_queue_handle_sink_event:<queue0> error: streaming stopped, reason not-linked (-1) [ WARN:0@0.339] global cap_gstreamer.cpp:2784 handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module filesrc0 reported: Internal data stream error. [ WARN:0@0.339] global cap_gstreamer.cpp:1199 startPipeline OpenCV | GStreamer warning: unable to start pipeline Number of audio samples: 0 Number of video frames: 10 [ WARN:0@0.339] global cap_gstreamer.cpp:1164 isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created ``` 6. Play an audio stream with seeking AFTER the fix: ```bash $ ./videocapture_audio_combination --audio "filesrc location=test1.mkv ! queue ! matroskademux name=demux demux.audio_0 ! decodebin ! audioconvert ! appsink"CAP_PROP_AUDIO_DATA_DEPTH: CV_16S CAP_PROP_AUDIO_SAMPLES_PER_SECOND: 48000 CAP_PROP_AUDIO_TOTAL_CHANNELS: 2 CAP_PROP_AUDIO_TOTAL_STREAMS: [ WARN:0@0.025] global cap_gstreamer.cpp:1903 getProperty OpenCV | GStreamer: CAP_PROP_AUDIO_TOTAL_STREAMS property is not supported 0 Timestamp: 0.0000 Timestamp: 24.0000 Timestamp: 48.0000 Timestamp: 72.0000 Timestamp: 96.0000 Timestamp: 120.0000 Timestamp: 144.0000 Timestamp: 168.0000 Timestamp: 192.0000 Timestamp: 216.0000 Timestamp: 3500.0000 Timestamp: 3504.0000 Timestamp: 3528.0000 Timestamp: 3552.0000 Timestamp: 3576.0000 Timestamp: 3600.0000 Timestamp: 3624.0000 Timestamp: 3648.0000 Timestamp: 3672.0000 Timestamp: 3696.0000 Timestamp: 3720.0000 Timestamp: 3744.0000 Timestamp: 3768.0000 Timestamp: 3792.0000 Timestamp: 3816.0000 Timestamp: 3840.0000 Timestamp: 3864.0000 Timestamp: 3888.0000 Timestamp: 3912.0000 Timestamp: 3936.0000 ``` 7. Play a video stream with seeking AFTER the fix: ```bash $ ./videocapture_audio_combination --audio "filesrc location=Jellyfish_360_10s_5MB.mkv ! queue ! matroskademux name=demux demux.video_0 ! decodebin ! videoconvert ! video/x-raw, format=BGR ! appsink drop=1" [ WARN:0@0.033] global cap_gstreamer.cpp:1746 open OpenCV | GStreamer warning: Cannot query video position: status=1, value=22, duration=300 CAP_PROP_AUDIO_DATA_DEPTH: CV_16S CAP_PROP_AUDIO_SAMPLES_PER_SECOND: 44100 CAP_PROP_AUDIO_TOTAL_CHANNELS: 0 CAP_PROP_AUDIO_TOTAL_STREAMS: [ WARN:0@0.034] global cap_gstreamer.cpp:1903 getProperty OpenCV | GStreamer: CAP_PROP_AUDIO_TOTAL_STREAMS property is not supported 0 Timestamp: 0.0000 Timestamp: 33.0000 Timestamp: 67.0000 Timestamp: 100.0000 Timestamp: 133.0000 Timestamp: 167.0000 Timestamp: 200.0000 Timestamp: 233.0000 Timestamp: 267.0000 Timestamp: 300.0000 0:00:00.335931693 3893501 0x55bbe76ad920 WARN matroskareadcommon matroska-read-common.c:759:gst_matroska_read_common_parse_skip:<demux:sink> Unknown CueTrackPositions subelement 0xf0 - ignoring 0:00:00.335952823 3893501 0x55bbe76ad920 WARN matroskareadcommon matroska-read-common.c:759:gst_matroska_read_common_parse_skip:<demux:sink> Unknown CueTrackPositions subelement 0xf0 - ignoring 0:00:00.335988029 3893501 0x55bbe76ad920 WARN basesrc gstbasesrc.c:1742:gst_base_src_perform_seek:<filesrc0> duplicate event found 184 Timestamp: 3467.0000 Timestamp: 3500.0000 Timestamp: 3533.0000 Timestamp: 3567.0000 Timestamp: 3600.0000 Timestamp: 3633.0000 Timestamp: 3667.0000 Timestamp: 3700.0000 Timestamp: 3733.0000 Timestamp: 3767.0000 Timestamp: 3800.0000 Timestamp: 3833.0000 Timestamp: 3867.0000 Timestamp: 3900.0000 Timestamp: 3933.0000 Timestamp: 3967.0000 Timestamp: 4000.0000 Timestamp: 4033.0000 Timestamp: 4067.0000 Timestamp: 4100.0000 ```

asmorkalov self-requested a review September 8, 2023 08:36

asmorkalov added the category: videoio label Sep 8, 2023

asmorkalov added this to the 4.9.0 milestone Sep 8, 2023

asmorkalov self-assigned this Sep 8, 2023

kecsap force-pushed the 4.x branch from 96c7bac to 921ee30 Compare September 13, 2023 09:00

asmorkalov reviewed Sep 13, 2023

View reviewed changes

modules/videoio/test/test_video_io.cpp Show resolved Hide resolved

kecsap force-pushed the 4.x branch from 921ee30 to 0915d68 Compare September 20, 2023 20:46

opencv-alalek requested a review from asmorkalov September 27, 2023 04:58

asmorkalov reviewed Oct 5, 2023

View reviewed changes

modules/videoio/src/cap_gstreamer.cpp Show resolved Hide resolved

modules/videoio/src/cap_gstreamer.cpp Show resolved Hide resolved

modules/videoio/src/cap_gstreamer.cpp Outdated Show resolved Hide resolved

kecsap force-pushed the 4.x branch from 0915d68 to f03f5ed Compare October 9, 2023 19:51

opencv-alalek added the bug label Oct 10, 2023

asmorkalov requested a review from mshabunin October 13, 2023 06:46

asmorkalov approved these changes Oct 19, 2023

View reviewed changes

mshabunin reviewed Oct 19, 2023

View reviewed changes

kecsap force-pushed the 4.x branch from f03f5ed to 648411d Compare October 20, 2023 14:43

mshabunin reviewed Oct 24, 2023

View reviewed changes

modules/videoio/src/cap_gstreamer.cpp Show resolved Hide resolved

modules/videoio/src/cap_gstreamer.cpp Outdated Show resolved Hide resolved

kecsap force-pushed the 4.x branch from 648411d to bb953df Compare October 27, 2023 10:47

mshabunin reviewed Oct 29, 2023

View reviewed changes

kecsap force-pushed the 4.x branch from bb953df to 7961aa2 Compare November 2, 2023 20:45

kecsap force-pushed the 4.x branch from 7961aa2 to 7f53fc4 Compare November 2, 2023 20:49

asmorkalov requested a review from mshabunin November 3, 2023 06:13

mshabunin approved these changes Nov 7, 2023

View reviewed changes

videoio: moved GStreamer-specific test to test_gstreamer.cpp

1b26ca0

mshabunin force-pushed the 4.x branch from ee2a6ab to 1b26ca0 Compare November 8, 2023 08:53

asmorkalov merged commit b1e0c4d into opencv:4.x Nov 8, 2023
26 checks passed

asmorkalov added a commit to asmorkalov/opencv that referenced this pull request Nov 8, 2023

Fixed build warning introduced in opencv#24243.

6b0e774

asmorkalov mentioned this pull request Nov 8, 2023

Fixed build warning introduced in #24243 #24512

Merged

6 tasks

asmorkalov added a commit to asmorkalov/opencv that referenced this pull request Nov 8, 2023

Fixed build warning introduced in opencv#24243.

b203c7c

asmorkalov added a commit that referenced this pull request Nov 8, 2023

Merge pull request #24512 from asmorkalov:as/gstreamer_warn_fix

41c335e

Fixed build warning introduced in #24243

IskXCr pushed a commit to Haosonn/opencv that referenced this pull request Dec 20, 2023

Fixed build warning introduced in opencv#24243.

bdf7b48

thewoz pushed a commit to thewoz/opencv that referenced this pull request Jan 4, 2024

Fixed build warning introduced in opencv#24243.

cf81fb8

asmorkalov mentioned this pull request Jan 19, 2024

5.x merge 4.x #24862

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix gstreamer backend with manual pipelines #24243

Fix gstreamer backend with manual pipelines #24243

kecsap commented Sep 7, 2023 •

edited

asmorkalov commented Sep 8, 2023

kecsap commented Sep 13, 2023 •

edited

asmorkalov commented Sep 13, 2023

kecsap commented Sep 13, 2023 •

edited

asmorkalov commented Sep 13, 2023

kecsap commented Sep 20, 2023

kecsap commented Sep 26, 2023

kecsap commented Oct 9, 2023 •

edited

kecsap commented Oct 17, 2023 •

edited

asmorkalov left a comment

kecsap commented Oct 19, 2023

mshabunin commented Oct 20, 2023

mshabunin left a comment

kecsap commented Oct 27, 2023

mshabunin left a comment

kecsap commented Nov 2, 2023 •

edited

mshabunin left a comment

Fix gstreamer backend with manual pipelines #24243

Fix gstreamer backend with manual pipelines #24243

Conversation

kecsap commented Sep 7, 2023 • edited

Pull Request Readiness Checklist

asmorkalov commented Sep 8, 2023

kecsap commented Sep 13, 2023 • edited

asmorkalov commented Sep 13, 2023

kecsap commented Sep 13, 2023 • edited

asmorkalov commented Sep 13, 2023

kecsap commented Sep 20, 2023

kecsap commented Sep 26, 2023

kecsap commented Oct 9, 2023 • edited

kecsap commented Oct 17, 2023 • edited

asmorkalov left a comment

Choose a reason for hiding this comment

kecsap commented Oct 19, 2023

mshabunin commented Oct 20, 2023

mshabunin left a comment

Choose a reason for hiding this comment

kecsap commented Oct 27, 2023

mshabunin left a comment

Choose a reason for hiding this comment

kecsap commented Nov 2, 2023 • edited

mshabunin left a comment

Choose a reason for hiding this comment

kecsap commented Sep 7, 2023 •

edited

kecsap commented Sep 13, 2023 •

edited

kecsap commented Sep 13, 2023 •

edited

kecsap commented Oct 9, 2023 •

edited

kecsap commented Oct 17, 2023 •

edited

kecsap commented Nov 2, 2023 •

edited