Qt VideoEditor #745

corranwebster · 2020-02-24T08:56:21Z

This is a viewer-only Qt VideoEditor. This only presents the video (provided as a URL), not any associated controls like play buttons, position sliders, etc. but you can bind traits to player state and then build a custom interface around them - the example gives a good idea of how to do this.

There are some glitches on MacOS retina displays which appear to be a problem with the underlying QtMultimedia implementation.

There is no Wx implementation. There are no tests yet.

JackDra · 2020-03-02T15:14:31Z

The video streaming seems to work fine both locally and with the demo video link on my mac.

The only issue I had was there is a weird background image displayed behind the video, its a black image half a cut off text field at the top of the video pane

JackDra · 2020-03-04T16:22:05Z

The video streaming seems to work fine both locally and with the demo video link on my mac.

The only issue I had was there is a weird background image displayed behind the video, its a black image half a cut off text field at the top of the video pane

Sorry this problem was fixed when I built with the latest version of pyqt5 instead of pyside2.

pyqt5 had to be installed from pip tho...... there were some QAudio attribute errors when using the edm version of pyqt5. going off this, there was a comment suggesting

make sure you are building with a Qt version >= 5.8

FEAT: QT VideoEditor pulling frames

jvkersch · 2020-11-25T17:42:26Z

Some quick drive-by comments: on Linux (Ubuntu 18.04), the player refused to play video with EDM-installed PyQt5 and PySide2, with the error message 'no service found for - "org.qt-project.qt.mediaplayer"'. Installing pyqt5 from PyPI, as Jack suggested, made the video playable, but ~~(a) it is rotated over 90 degrees, and (b)~~ playback speed is very slow. Likely not an issue with TraitsUI, but suggestions for debugging (if any) are welcome.

jvkersch · 2020-11-26T14:41:18Z

Some updates: @mdsmarte and I looked into running the video editor on Windows and Linux, and here are some findings. On Linux, the EDM-installed PyQt5 does not have the multimedia plugin and fails to play video. Using PyQt5 from PyPI instead works. As reported above, playback is very choppy both with local and remote videos. Commenting out the following line makes playback smooth again (comparable to playing the video in a native media player):

traitsui/traitsui/qt4/video_editor.py

Line 343 in 2b8620c

self.media_player.setPosition(int(self.position * 1000))

.

In Windows 10, I see the same kind of choppiness, with the same workaround. Playback works out of the box with both the EDM and the PyPI version of PyQt5. I did have to install the necessary codecs.

traitsui/qt4/video_editor.py

jwiggins · 2021-04-09T08:04:38Z

@aaronayres35 Weren't you seeing intermittent failures around the HTMLEditor tests? Is there an issue?

======================================================================
FAIL: test_open_external_link (traitsui.tests.editors.test_html_editor.TestHTMLEditor)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\a\traitsui\traitsui\.edm\envs\traitsui-test-3.6-pyside2\lib\site-packages\traitsui\tests\editors\test_html_editor.py", line 310, in test_open_external_link
    html_view.inspect(HTMLContent()),
AssertionError: 'External Link' not found in ''

jwiggins

@corranwebster I'm happy to finish this PR but I have several questions first.

traitsui/qt4/video_editor.py

jwiggins · 2021-04-09T11:04:43Z

traitsui/qt4/video_editor.py

+    # Signal handlers -------------------------------------------------------
+
+    def _state_changed_emitted(self, state):
+        self.state = reversed_state_map[state]


Should this be wrapped by a self.updating_value() context?

Possibly - again, don't have the context loaded into my head, so it you think it is needed, do it!

I'm leaving this alone for now. It's not as straightforward as position was.

jwiggins · 2021-04-09T11:08:07Z

traitsui/examples/demo/Standard_Editors/VideoEditor_demo.py

+                state='state',
+                position='position',
+                duration='duration',
+                video_error='error',
+                media_status='status',
+                buffer='buffer',
+                muted='muted',
+                volume='volume',
+                playback_rate='playback_rate',
+                image_fun='image_fun'


It would be nice not to require users to provide traits for all of this editor state. How do the trait declarations in VideoEditor need to change?

This editor was designed to be low-level and makes heavy use of the sync_name metadata, behaviour is explained here:

traitsui/traitsui/editor.py

Lines 594 to 617 in 7f3dbaf

Initializes and synchronizes (as needed) editor traits with the

value of corresponding factory traits. The name of the factory

trait and the editor trait must match and the factory trait needs

to have ``sync_value`` metadata set. The strategy followed is:

- for each factory trait with ``sync_value`` metadata:

1. if the value is a :class:`ContextValue` instance then

call :meth:`sync_value` with the ``name`` from the

context value.

2. if the trait has ``sync_name`` metadata, look at the

referenced trait value and if it is a non-empty string

then use this value as the name of the value in the

context.

3. otherwise initialize the current value of the factory

trait to the corresponding value of the editor.

- synchronization mode in cases 1 and 2 is taken from the

``sync_value`` metadata of the editor trait first and then

the ``sync_value`` metadata of the factory trait if that is

empty.

- if the value is a container type, then the `is_list` metadata

is set to

In particular users of the code are responsible for setting up any play/pause buttons, muting etc. If they are left empty, then the default values from the Editor will be used, I think, so we should make sure that those are sensible.

If we want to allow constant values (eg. setting muted to True and not synchronising), then we might need to have muted and muted_name traits.

If they are left empty, then the default values from the Editor will be used

This is true, but since the default values are empty strings it causes a TraitError if nothing is passed.

I don't understand the mechanisms in TraitsUI well enough to change this [even after some experimentation], so I'm going to leave it alone in this PR.

traitsui/examples/demo/Standard_Editors/VideoEditor_demo.py

corranwebster · 2021-04-12T07:43:32Z

Yes, feel free to run with this as needed (time permitting), making whatever changes you think make sense - it may make sense to strip out the image transform stuff, particularly if it affects performance.

jwiggins · 2021-04-12T11:40:15Z

This is ready for review if someone would like to take a look

aaronayres35 · 2021-04-12T14:22:21Z

@aaronayres35 Weren't you seeing intermittent failures around the HTMLEditor tests? Is there an issue?

======================================================================
FAIL: test_open_external_link (traitsui.tests.editors.test_html_editor.TestHTMLEditor)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\a\traitsui\traitsui\.edm\envs\traitsui-test-3.6-pyside2\lib\site-packages\traitsui\tests\editors\test_html_editor.py", line 310, in test_open_external_link
    html_view.inspect(HTMLContent()),
AssertionError: 'External Link' not found in ''

Ah yeah I was, although they weren't on this specific test IIRC. Issue was #1551 which was closed by #1553 (although that may have jumped the gun and perhaps it needs to be re-opened)

jwiggins · 2021-04-12T14:34:40Z

Hmm... Yeah. Speculative fixes can be risky.

JackDra · 2021-04-12T14:34:53Z

Yes, feel free to run with this as needed (time permitting), making whatever changes you think make sense - it may make sense to strip out the image transform stuff, particularly if it affects performance.

If there is more time available (which sounds like no), I had thought about having both video player and image transformed video player coded up, and if the user does not pass an image transformation function, it will fall back to the regular video player implementation.

jwiggins · 2021-04-12T14:43:15Z

if the user does not pass an image transformation function, it will fall back to the regular video player implementation.

I'm pretty sure that's already implemented.

rahulporuri · 2021-04-12T15:52:58Z

@jwiggins can you please update the PR description - which probably is out-of-date.

jwiggins · 2021-04-12T16:15:19Z

I'm not sure what should be changed in the description

rahulporuri

Mostly LGTM with a bunch of comments.

I guess the video editor isn't ready for prime time yet so we will keep it out of traitsui.api for now. When we do decide to include it in the api module though, we will probably want to also include the video editor-specific traits defined in the factory module.

traitsui/editors/video_editor.py

traitsui/examples/demo/Standard_Editors/VideoEditor_demo.py

traitsui/qt4/video_editor.py

rahulporuri · 2021-04-13T12:20:13Z

traitsui/qt4/video_editor.py

+    # Editor interface
+    # ------------------------------------------------------------------------
+
+    def init(self, parent):


We're passing in the parent but we are never using it. Not sure if we need to be though.

Seems to be a common pattern in TraitsUI Qt code...

rahulporuri · 2021-04-13T12:21:34Z

traitsui/qt4/video_editor.py

+        self.media_player.setVideoOutput(self.control)
+
+    def update_to_functional(self):
+        self.control = ImageWidget(image_fun=self.image_fun)


should we be setting widget here like we are with VideoSurface?

I'm not sure what you're asking. VideoSurface takes a widget argument, which is equal to self.control in this method.

traitsui/tests/editors/test_video_editor.py

rahulporuri · 2021-04-13T12:42:20Z

Update : This doesn't work as expected with pyside2 either

I LGTM'ed too early. I ran the example on windows and it isnt working as expected.

win + cp36 + pyqt5 5.14.2-4 and qt 5.12.6-10

# Packages in environment 'traitsui-test-3.6-pyqt5'
#  prefix: 'C:\Users\rporuri\.edm\envs\traitsui-test-3.6-pyqt5'
appdirs              1.4.3-1        enthought/free
apptools             5.1.0-1        enthought/free
bzip2_runtime        1.0.8-1        enthought/free
chaco                4.8.0-7        enthought/free
configobj            5.0.6.post9-1  enthought/free
coverage             4.3.4-1        enthought/free
distribute_remove    1.0.0-4        enthought/free
enable               5.1.0-1        enthought/free
flake8               3.8.4-3        enthought/free
flake8_ets           1.0.0-1        enthought/free
fonttools            3.29.1-1       enthought/free
freetype             2.10.0-7       enthought/free
h5py                 2.10.0-3       enthought/free
harfbuzz             2.6.4-4        enthought/free
icu4c                50.2-2         enthought/free
importlib_metadata   2.0.0-1        enthought/free
importlib_resources  3.3.0-1        enthought/free
libblosc             1.20.1-2       enthought/free
liblcms2             2.11-1         enthought/free
liblz4               1.9.2-1        enthought/free
libopenjpeg          2.4.0-2        enthought/free
libpng               1.6.37-1       enthought/free
libtiff              4.0.10-9       enthought/free
libzstd              1.4.8-1        enthought/free
mccabe               0.6.1-1        enthought/free
mkl                  2018.0.3-2     enthought/free
numexpr              2.6.9-3        enthought/free
numpy                1.17.4-1       enthought/free
packaging            16.8-4         enthought/free
pandas               0.25.3-3       enthought/free
pillow               7.2.0-2        enthought/free
pip                  20.0.2-3       enthought/free
pycodestyle          2.6.0-1        enthought/free
pyface               7.3.0-1        enthought/free
pyflakes             2.2.0-1        enthought/free
pygments             2.2.0-1        enthought/free
pyparsing            2.2.0-1        enthought/free
pyqt5                5.14.2-4       enthought/free
pyqt5_sip            12.7.2-1       enthought/free
pytables             3.5.1-7        enthought/free
python_dateutil      2.8.0-4        enthought/free
pytz                 2020.1-1       enthought/free
qt                   5.12.6-10      enthought/free
setuptools           41.6.0-4       enthought/free
six                  1.15.0-1       enthought/free
traits               6.2.0-1        enthought/free
traitsui             7.1.1-4        enthought/free
zipp                 3.4.0-2        enthought/free
zlib_runtime         1.2.11-1       enthought/free

Note that I also see the following on the command line when i run the example

>python VideoEditor_demo.py
DirectShowPlayerService::doRender: Unresolved error code 0x80040218 (IDispatch error #24)
DirectShowPlayerService::doRender: Unresolved error code 0x80040218 (IDispatch error #24)

jwiggins · 2021-04-13T12:56:53Z

@rahulporuri I believe you need to install codecs on Windows in order for Qt Multimedia to work properly.

JackDra · 2021-04-15T15:54:33Z

One last comment that is not necessary:
It may be good to know on what platform, what movie formats are supported (with what extension tag).

We had an issue in the past where our vp9 formatted files were not playing. An example file:
https://file-examples-com.github.io/uploads/2020/03/file_example_WEBM_480_900KB.webm

It might have been that our file did not actually have an extension .webm and QTMediaPlayer determines the format through the extension.

jwiggins · 2021-04-16T10:43:55Z

Good suggestion @JackDra. I'll do that here for now and we can move the info to the appropriate place later. Most of this info was collected by @mdsmarte, but was attached to a private issue.

Qt Wiki: Multimedia Backends

Windows:
Depending on how Qt was built [and possibly environment variables], videos are handled by either WMF or DirectShow. If using DirectShow, a user can increase their codec and container-file capabilities by installing the K-Lite Codec Pack. WMF is [currently] the future of Windows media handling, but the Qt5 supplied by EDM only has DirectShow support.

MacOS:
If it's supported by AVFoundation, you can open it.

Linux:
Make sure GStreamer is installed.

jwiggins · 2021-04-16T11:34:49Z

@rahulporuri If you have no further objections, I'll merge this today.

rahulporuri · 2021-04-16T11:37:04Z

@rahulporuri If you have no further objections, I'll merge this today.

Please go ahead.

Most of this info was collected by @mdsmarte, but was attached to a private issue ...

We can move this information into the user documentation separately.

@jwiggins are you hoping for a new traitsui release soon?

rahulporuri · 2021-04-16T12:01:09Z

UPDATE : Please ignore. This is the whole point of the custom image_func function. Sigh.

~~good news : after installing the k-lite basic codec pack, the video shows up on my machine (windows + pyqt5 from edm + py36)~~

~~bad news : the video layout is screwed up~~

Rotated video

![well-that-is-unfornatute](https://user-images.githubusercontent.com/1926457/115021371-6968bc80-9eab-11eb-9e6b-3d6ca21392a6.gif)

rahulporuri · 2021-04-16T12:05:25Z

We had an issue in the past where our vp9 formatted files were not playing. An example file:
https://file-examples-com.github.io/uploads/2020/03/file_example_WEBM_480_900KB.webm

~~I'm seeing the same behavior with this file as well - the video is rotated 90 deg anti clockwise and left-right are flipped.~~
Again, this was the intended behavior.

jwiggins · 2021-04-16T12:06:52Z

The rotation is the expected behavior. It's showing that users can manipulate the stream via the image_func trait.

rahulporuri · 2021-04-16T12:09:19Z

thanks a lot @jwiggins for pushing this across the finish line. please go ahead and merge.

jwiggins · 2021-04-16T12:09:50Z

Thanks everyone, this is a nice addition to the library.

rahulporuri · 2021-04-16T12:10:58Z

are you hoping for a new traitsui release soon?

@jwiggins ^ . We are planning on making a release early Feb. We can make one sooner if that's useful.

jwiggins · 2021-04-16T12:14:03Z

Talk to @mdsmarte and/or @ffishburn. Their customer project is an intended user of this feature.

corranwebster and others added 2 commits February 21, 2020 14:09

Initial commit of Qt VideoEditor implementation.

cfbc08f

FEAT: broken implementation to pull frames from video stream

5e415e7

JackDra added 5 commits March 3, 2020 14:41

temp

347c350

Moved PIL and image function dependancy out to demo for now

f73a927

Now defaults to QVideoWidget, unless image_fun is present

b5f4428

Docs

31b60eb

FEAT: option to toggle antialising when resizing image to frame box.

ed08705

JackDra mentioned this pull request Mar 4, 2020

FEAT: QT VideoEditor pulling frames #753

Merged

Merge pull request #753 from enthought/feat/video-editor-pull-frames

2b8620c

FEAT: QT VideoEditor pulling frames

jwiggins reviewed Apr 6, 2021

View reviewed changes

traitsui/qt4/video_editor.py Outdated Show resolved Hide resolved

jwiggins added 6 commits April 8, 2021 14:04

Merge branch 'master' into feat/video-editor

adc5dcb

Add a simple test for VideoEditor

39d877a

Fix some style-check failures

2a9b2e9

More VideoEditor test fixes

7556169

Fix copyright headers

013a05c

Cleanups; Avoid unnecessary setPosition calls

1dc135e

jwiggins reviewed Apr 9, 2021

View reviewed changes

PR feedback

b2aea11

aaronayres35 mentioned this pull request Apr 12, 2021

Intermittent failures in traitsui.tests.editors.test_html_editor.py #1551

Open

Fix arg list issue and segfault

f0d0412

rahulporuri self-requested a review April 12, 2021 15:50

rahulporuri changed the title ~~[WIP] Qt VideoEditor~~ Qt VideoEditor Apr 12, 2021

rahulporuri approved these changes Apr 13, 2021

View reviewed changes

jwiggins added 2 commits April 13, 2021 15:24

PR feedback

495dbe1

Use @observe in VideoEditor

2a86f9f

Use observation in the video editor demo

d96cdb9

jwiggins merged commit 616b121 into master Apr 16, 2021

jwiggins deleted the feat/video-editor branch April 16, 2021 12:09

mdsmarte mentioned this pull request Apr 26, 2021

Expose notify interval in VideoEditor #1609

Merged

This was referenced Apr 29, 2021

Document new VideoEditor in Advanced Trait Editors #1629

Closed

User facing docs for VideoEditor #1630

Merged

	Initializes and synchronizes (as needed) editor traits with the
	value of corresponding factory traits. The name of the factory
	trait and the editor trait must match and the factory trait needs
	to have ``sync_value`` metadata set. The strategy followed is:

	- for each factory trait with ``sync_value`` metadata:

	1. if the value is a :class:`ContextValue` instance then
	call :meth:`sync_value` with the ``name`` from the
	context value.
	2. if the trait has ``sync_name`` metadata, look at the
	referenced trait value and if it is a non-empty string
	then use this value as the name of the value in the
	context.
	3. otherwise initialize the current value of the factory
	trait to the corresponding value of the editor.

	- synchronization mode in cases 1 and 2 is taken from the
	``sync_value`` metadata of the editor trait first and then
	the ``sync_value`` metadata of the factory trait if that is
	empty.

	- if the value is a container type, then the `is_list` metadata
	is set to

Qt VideoEditor #745

Qt VideoEditor #745

Conversation

corranwebster commented Feb 24, 2020

JackDra commented Mar 2, 2020

JackDra commented Mar 4, 2020 • edited

jvkersch commented Nov 25, 2020 • edited

jvkersch commented Nov 26, 2020

jwiggins commented Apr 9, 2021

jwiggins left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

corranwebster commented Apr 12, 2021

jwiggins commented Apr 12, 2021

aaronayres35 commented Apr 12, 2021

jwiggins commented Apr 12, 2021

JackDra commented Apr 12, 2021

jwiggins commented Apr 12, 2021

rahulporuri commented Apr 12, 2021

jwiggins commented Apr 12, 2021

rahulporuri left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rahulporuri commented Apr 13, 2021 • edited

jwiggins commented Apr 13, 2021

JackDra commented Apr 15, 2021

jwiggins commented Apr 16, 2021

jwiggins commented Apr 16, 2021

rahulporuri commented Apr 16, 2021 • edited

rahulporuri commented Apr 16, 2021 • edited

rahulporuri commented Apr 16, 2021 • edited

jwiggins commented Apr 16, 2021

rahulporuri commented Apr 16, 2021

jwiggins commented Apr 16, 2021

rahulporuri commented Apr 16, 2021

jwiggins commented Apr 16, 2021

JackDra commented Mar 4, 2020 •

edited

jvkersch commented Nov 25, 2020 •

edited

rahulporuri commented Apr 13, 2021 •

edited

rahulporuri commented Apr 16, 2021 •

edited

rahulporuri commented Apr 16, 2021 •

edited

rahulporuri commented Apr 16, 2021 •

edited