FFmpeg cap consider rotation metadata 3.4 #17489

Lapshin · 2020-06-06T11:02:32Z

Pull Request Readiness Checklist

Relates to #15499

Duplicates commit of #17452

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

I agree to contribute to the project under OpenCV (BSD) License.
To the best of my knowledge, the proposed patch is not based on a code under GPL or other license that is incompatible with OpenCV
The PR is proposed to proper branch
There is reference to original bug report and related work
There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
The feature is well documented and sample code can be built with the project CMake

allow_multiple_commits=1

asmorkalov

Please take a look on https://pullrequest.opencv.org/buildbot/builders/precommit_windows64/builds/24964/steps/compile%20release/logs/stdio

asmorkalov · 2020-06-08T07:52:40Z

modules/videoio/src/cap_ffmpeg.cpp

+            return;
+        }
+        int angle = ffmpegCapture->rotation_angle;
+        if(!ffmpegCapture->rotation_auto || (angle % 360) == 0) {


OpenCV build for Windows used FFmpeg through dedicated wrapper DLL and internal fields are not visible. You have to use getProperty method to get values.

Lapshin · 2020-06-08T22:35:07Z

Is it allowed to change the declaration of cvRetrieveFrame_FFMPEG and CvRetrieveFrame_Plugin?

So, now it able to build, but some tests fail. I do not know where I can get testing video samples and how to run the tests.
Maybe I should also change tests because now image rotates automatically...

Please give me the introduction info about this

asmorkalov · 2020-06-09T07:43:06Z

@Lapshin Thanks for the fix. I cherry-picked test for the new rotation test from #17500 and pushed to your branch. I'll check the tests status on CI and let you know.

modules/videoio/test/test_ffmpeg.cpp

asmorkalov · 2020-06-09T10:30:01Z

i checked FFmpeg version and it looks like the rotation metadata is not supported by libav in Ubuntu 14.04. It's used for 3.4 testing. It looks like we need another branch inside FFmpeg code or skip the test for old systems. I'll check and come up with solution in mean time.

Lapshin · 2020-06-12T10:57:30Z

Hi @asmorkalov,

You can delegate it to me if you have not enough time

asmorkalov · 2020-06-18T10:29:36Z

@Lapshin Sorry for delay. I investigated FFmpeg API and options to support libAV on Ubuntu 14.04. It looks like public API for ROTATIONMATRIX appeared in FFmpeg (libAV) in 2014 and is not present in the old Ubuntu. I'm discussing options with alalek and team members and return back in mean time.

asmorkalov · 2020-06-22T10:59:30Z

@Lapshin @alalek I pushed extra checks to test and code itself to handle old FFmpeg correctly. Please take a look and comment.

modules/videoio/src/cap_ffmpeg_impl.hpp

Lapshin · 2020-06-23T07:49:10Z

Your changes are good for me
But I can't understand why the same tests fail on windows

asmorkalov · 2020-06-23T10:39:07Z

Ci build for Windows does not re-build FFmpeg warpper each time, but use old version. We added extra fields to classes and it leads to binary incompatibly. I'm looking on the issue right now.

modules/videoio/src/cap_ffmpeg.cpp

asmorkalov · 2020-07-17T08:28:25Z

@Lapshin I moved rotation step to cv::VideoCapture implementation for FFmpeg and restored FFmpeg wrapper interface for binary compatibility on Windows. Please check the latest version on your side.

Lapshin · 2020-07-20T12:16:30Z

Thank you, @asmorkalov. You did it :)

asmorkalov · 2020-07-20T16:02:58Z

@alalek The patch is ready for integration with squash. I fixed Windows test failure and related warnings. Windows builds will report "not supported" till FFmpeg wrapper rebuild.

asmorkalov · 2020-07-22T13:19:28Z

@mshabunin Could you look at it.

mshabunin · 2020-07-22T16:37:24Z

@asmorkalov , could you please squash commits into two to preserve authors.

- Add VideoCapture camera orientation property for mp4 videos with camera orientation meta. - Add auto rotation for 90, 180, 270 degrees using cv::rotate

- Added test for automated rotation for MP4 videos with metadata - Fix 180 degrees rotation bug - Moved rotation logic to cv::VideoCapture implementation for FFmpeg and restore binary compatibility with FFmpeg wrapper.

asmorkalov · 2020-07-23T06:56:32Z

@mshabunin Done.

mshabunin · 2020-07-23T08:39:05Z

@asmorkalov , thank you. Do we need to port this feature to master branch?

asmorkalov · 2020-07-23T08:49:54Z

Yes, I expect that it can be merged as is with regular 3.4->master merge. Please let me know, if manual port is needed. I'll do it.

alalek · 2020-09-03T21:21:03Z

modules/videoio/src/cap_ffmpeg.cpp

+            return;
+        }
+
+        cv::rotate(mat, mat, flag);


cv::Mat / cv::rotate is not available for FFmpeg wrapper on 3.4 branch.
Build of Windows wrapper is broken

asmorkalov reviewed Jun 8, 2020

View reviewed changes

asmorkalov added the category: video label Jun 8, 2020

asmorkalov mentioned this pull request Jun 8, 2020

Add VideoCapture camera orientation property for mp4 videos with camera orientation meta #17452

Closed

6 tasks

Lapshin force-pushed the ffmpeg_cap_consider_rotation_metadata_3.4 branch 2 times, most recently from d52c1b9 to 60ecd75 Compare June 8, 2020 21:38

Lapshin commented Jun 9, 2020

View reviewed changes

modules/videoio/test/test_ffmpeg.cpp Show resolved Hide resolved

Lapshin commented Jun 22, 2020

View reviewed changes

modules/videoio/src/cap_ffmpeg_impl.hpp Outdated Show resolved Hide resolved

asmorkalov force-pushed the ffmpeg_cap_consider_rotation_metadata_3.4 branch from 32912e8 to 7423983 Compare June 22, 2020 16:00

asmorkalov self-requested a review July 13, 2020 11:31

asmorkalov requested changes Jul 13, 2020

View reviewed changes

modules/videoio/src/cap_ffmpeg.cpp Outdated Show resolved Hide resolved

asmorkalov self-requested a review July 20, 2020 16:01

asmorkalov approved these changes Jul 20, 2020

View reviewed changes

asmorkalov changed the title ~~Ffmpeg cap consider rotation metadata 3.4~~ FFmpeg cap consider rotation metadata 3.4 Jul 21, 2020

mshabunin approved these changes Jul 22, 2020

View reviewed changes

mshabunin self-assigned this Jul 22, 2020

Alexey Lapshin and others added 2 commits July 23, 2020 09:27

Autorotation for mp4 streams with metadata

f0271e5

- Add VideoCapture camera orientation property for mp4 videos with camera orientation meta. - Add auto rotation for 90, 180, 270 degrees using cv::rotate

MP4 autorotation tests and various fixes for Windows

7ed37b3

- Added test for automated rotation for MP4 videos with metadata - Fix 180 degrees rotation bug - Moved rotation logic to cv::VideoCapture implementation for FFmpeg and restore binary compatibility with FFmpeg wrapper.

asmorkalov force-pushed the ffmpeg_cap_consider_rotation_metadata_3.4 branch from 52c8ca3 to 7ed37b3 Compare July 23, 2020 06:34

opencv-pushbot merged commit 5bfa43f into opencv:3.4 Jul 23, 2020

mshabunin added the port into 4.x is needed Label for maintainers. Authors of PR can ignore this label Jul 23, 2020

alalek mentioned this pull request Jul 28, 2020

Merge 3.4 #17970

Merged

alalek reviewed Sep 3, 2020

View reviewed changes

alalek mentioned this pull request Sep 4, 2020

videoio: repair build of FFmpeg Windows wrapper #18265

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

FFmpeg cap consider rotation metadata 3.4 #17489

FFmpeg cap consider rotation metadata 3.4 #17489

Lapshin commented Jun 6, 2020 •

edited by mshabunin

asmorkalov left a comment

asmorkalov Jun 8, 2020

Lapshin commented Jun 8, 2020

asmorkalov commented Jun 9, 2020

asmorkalov commented Jun 9, 2020

Lapshin commented Jun 12, 2020

asmorkalov commented Jun 18, 2020

asmorkalov commented Jun 22, 2020

Lapshin commented Jun 23, 2020

asmorkalov commented Jun 23, 2020

asmorkalov commented Jul 17, 2020

Lapshin commented Jul 20, 2020

asmorkalov commented Jul 20, 2020

asmorkalov commented Jul 22, 2020

mshabunin commented Jul 22, 2020

asmorkalov commented Jul 23, 2020

mshabunin commented Jul 23, 2020

asmorkalov commented Jul 23, 2020

alalek Sep 3, 2020

FFmpeg cap consider rotation metadata 3.4 #17489

FFmpeg cap consider rotation metadata 3.4 #17489

Conversation

Lapshin commented Jun 6, 2020 • edited by mshabunin

Pull Request Readiness Checklist

asmorkalov left a comment

Choose a reason for hiding this comment

asmorkalov Jun 8, 2020

Choose a reason for hiding this comment

Lapshin commented Jun 8, 2020

asmorkalov commented Jun 9, 2020

asmorkalov commented Jun 9, 2020

Lapshin commented Jun 12, 2020

asmorkalov commented Jun 18, 2020

asmorkalov commented Jun 22, 2020

Lapshin commented Jun 23, 2020

asmorkalov commented Jun 23, 2020

asmorkalov commented Jul 17, 2020

Lapshin commented Jul 20, 2020

asmorkalov commented Jul 20, 2020

asmorkalov commented Jul 22, 2020

mshabunin commented Jul 22, 2020

asmorkalov commented Jul 23, 2020

mshabunin commented Jul 23, 2020

asmorkalov commented Jul 23, 2020

alalek Sep 3, 2020

Choose a reason for hiding this comment

Lapshin commented Jun 6, 2020 •

edited by mshabunin