cv::VideoCapture ignores orientation metadata #15499
Comments
|
Do you have a recipe of how to create a video with this transformation? |
|
before: after:
|
|
You know, mediainfo also doesn't swap width and height when rotation metadata is present. After: Width and height of the new video are the same. There is a new Rotation parameter though. |
|
I see two ways here:
The second approach is simpler. |
|
I would go with the second approach. It's not only simpler, it's saner as well. |
I prefer the first option enabled by default. ffmpeg performs frame extraction with auto-rotation enabled, and the first option will be consistent with ffmpeg's. BTW, most video players treated the rotation parameter transparently, and the first option is more transparent in a sense. Another aspect, with opencv's videowriter, there is no flag for the rotation parameter. If we go with the first option, it will be simpler. |
asmorkalov
added
the
Hackathon
asmorkalov
changed the title
|
Action items for Hackathon:
|
|
Instead of cap_prop_auto_rotation we can use cap_Frame(position parameters) to change the rotation along the rotated parameters . |
|
+1 Any update on this feature? Using imwrite() with VideoCapture results in images that are not the same orientation as the video. |
|
Hi there, I faced the same problem, and have solutions. I have only few hours experience with opencv and ffmpeg, and do not sure that the code written with optimal algorithms The timings for sample program which opens video file and writes it to another file frame by frame: no-rotation: https://github.com/Lapshin/opencv/tree/ffmpeg_cap_consider_rotation_metadata__ffmpeg https://github.com/Lapshin/opencv/tree/ffmpeg_cap_consider_rotation_metadata__opencv tested with: |
|
difference between solutions in implementation: |
|
lol, I made the measurements on the overloaded cpu :) the new test times: https://github.com/Lapshin/opencv/tree/ffmpeg_cap_consider_rotation_metadata__ffmpeg https://github.com/Lapshin/opencv/tree/ffmpeg_cap_consider_rotation_metadata__opencv So, I'm confused about these timings... |
|
@Lapshin Thanks for your effort. OpenCV core provides |
|
@asmorkalov thank you for this clarification. is it ok to make PR based on the master branch? |
|
I suppose to rebase the PR to 3.4. We merge 3.4 to master regularly and fixes will appear in both branches. |
|
Done |
|
@Lapshin @asmorkalov hi I tried to use the proposed cv2 video rotation properties but they are not present in cv2 video capture objects. cv2.__version__
Out[4]: '4.4.0'
cap.get(cv2.CAP_PROP_FRAME_COUNT)
Out[5]: 31.0
cap.get(cv2.CAP_PROP_AUTO_ROTATION)
Traceback (most recent call last):
File "/Users/glennjocher/PycharmProjects/yolov5/venv/lib/python3.8/site-packages/IPython/core/interactiveshell.py", line 3417, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-3-f14478ca3566>", line 1, in <module>
self.cap.get(cv2.CAP_PROP_AUTO_ROTATION)
AttributeError: module 'cv2.cv2' has no attribute 'CAP_PROP_AUTO_ROTATION'
cap.get(cv2.CAP_PROP_ROTATION) # same resultWhat is the current best practices method for handling portrait videos? I have two small test videos we use for YOLOv5 testing, one portrait one landscape, and we are investigating how to auto-detect the portrait orientation correctly: https://github.com/ultralytics/yolov5/releases/download/v1.0/decelera_landscape.mov Any help would be appreciated. Thanks! |
|
Hi @glenn-jocher, This fix works only if opencv uses ffmpeg as video processing backend I see changes under tag Regards |
|
@Lapshin oh, awesome! I'll wait for the |
Workaround for OpenCV bug, where orientation of video is not detected properly. opencv/opencv#15499
|
well, as far as I can tell, this is a total disaster. The default behaviour of opencv API changed, but there is no way to detect it (if the captured frame image was rotated or not). Trying to disable rotation to ensure consistent results with cv2.VideoCapture(filename,cv2.CAP_AUTO,(cv2.CAP_PROP_ORIENTATION_AUTO,False)) results in (I'm guessing because ffmpeg also needs to be at some specific version): [ERROR:0] global /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip-req-build-k95y01np/opencv/modules/videoio/src/cap_ffmpeg_impl.hpp (927) open VIDEOIO/FFMPEG: unsupported parameters in .open(), see logger INFO channel for details. Bailout This should absolutely have been done so that the existing code would not break (by not rotating frames by default, as before), but right now when I made this upgrade (using mac ports) Uninstalling opencv-python-4.4.0.44: all the existing code that used cv2.VideoCapture() broke on non-zero degrees rotated videos. And just going in and changing every place will not work because I cannot ensure the specific set of versions being present on all platforms where the code runs. |
|
I agree, default value should be false to not ruin software after update @asmorkalov, are you in agreement with it? |
|
Problem looks like the processing of undocumented / unsupported cases or exploiting bugs. Some projects maintains behavior of old releases through runtime policies (e.g, CMake policies) - but it is a maintenance hell and some changes still may missing accurate policies update.
See these threads about imread and orientation story: #7638 (feature PR) #6348 (raised issue) #7638 (PR with control option) This instruction may help:
This property is not supported as "open" property for now. |
|
I don't understand. Are you saying that this code: video = cv2.VideoCapture(filename) is somehow using the opencv in undocumented, unsupported way or exploiting bugs? If that is indeed the case, then I would say you guys need to write less bugs :) in opencv 4.4 it would always return a first frame of a video, unrotated. in opencv 4.5 it will return a rotated version of a frame if the video has rotation tag. Clearly, this is breaking backwards compatibility and there is absolutely no good reason to do it. |
|
Bug is not handling metadata in your |
|
Very funny. So the autorotation feature that is solely based on video metadata is, in fact, improper use and exploitation of opencv? |
|
The goal of OpenCV |
|
The rotation metadata is added to videos on mobile phones that can be easily used in either landscape or portrait mode, and this metadata has to be used by video players that want to play back the video correctly oriented on non-mobile devices. The video frame itself is always recorded the same way, the orientation tells in what position the recording device was held during recording. When playing back on mobile device, the rotation metadata can basically be ignored, as the user can easily rotate the device to the same orientation that was used during recording. On computer monitors the video player has to rotate the video so that it is played back in correct orientation without rotating the physical screen. If OpenCV aims to be video player (and there was me, thinking that it is Computer Vision library), then rotating the video based on metadata is basically a right thing to do, I am not arguing against that. I am just saying that the way the change was implemented breaks existing code. If that is in dev teams view a necessary break then I would expect that users get big warnings about such cases in release notes, and clear instructions with examples how to control the feature for reverting back to the old behaviour. In real life use across multiple platforms "stepping back to older version of FFmpeg" or any other package is not a viable long-term option. Meanwhile, I have patched my code just to check for opencv version, and if it is 4.5 or above I am assuming the frames are already rotated, but long-term I will probably be migrating to something that does not aim to be everything in one, plus video player. |
For some videos, using
get()to get VideoCapture's width/height property, the results are not correct (the width and height are swapped).I found that there is a
rotatetag in the video metadata (ref. link ffmpeg get orientation inverted), e.g.,Could this issue be considered in future releases?
The text was updated successfully, but these errors were encountered: