WebRTC Video viewer

[hu-o]

Salut !

OSC is amazing ! I'm trying to use it for live video control surfaces (Video mixing, VJing,...), so having the ability to have video views on the pages would be really great !

As very low latency is the key to these usages, I think WebRTC could be an option. I'm experimenting with https://www.medialooks.com/ndi, a little new software that takes NDI or SDI live video feeds and convert them to WebRTC.

Any chance to have that kind of feature implemented one day ? What could be the approach if I want to add that ?

Hugo

[jean-emmanuel]

I've done some tests, here's what seems to be possible:

• a server option to define RTSP urls (rtsp://...) and encoding options (framerate, resolution, quality)
• a Stream widget that displays the video stream

I don't think very low latency (as in milliseconds) is possible here, but hopefully decent latency is.

[hu-o]

A stream widget would be very cool indeed ! The server side could probably be handled by ffmpeg ? It should be possible to achieve 1-200ms latency on a local network.

What strike me about webRTC is that I always tough web pages / web interfaces would always be somewhat limited in term of latency performances for live monitoring of audio and video signals. With WebRTC I can have Chrome displaying the feed of the camera of another device with no perceivable latency (below 40ms) . This mean you could do entire software interface with OSC, including audio and video feeds monitoring.

[jean-emmanuel]

Yep, server-side ffmepg is what I had in mind, I just pushed a something to be tested if you can build the app from sources:

• set the server's video-streams option to a valid rtsp url
• create an Image widget and set its stream property to 0

note:

• video-streams: one or multiple rtsp urls separated by spaces, and (optionnal) settings: framerate=<int> and resolution=<int>x<int>
• stream: rtsp url index starting at 0.

[jean-emmanuel]

Maybe with a powerful computer you'll get decent latency, I can't tell with mine.

I'm not going to implement any audio handling in O-S-C. Actually I'm not even sure yet this stream experimentation should be included in the end. Audio and video both come with specific performance concerns I'm not willing to address: separation of concerns is the key, O-S-C does the controls, not audio/video processing.

[hu-o]

Wow, that was fast !

Can't wait to try that, give me a day or two.

[hu-o]

Yes, I hunderstand the concern. I think this is not out of context. For live perforances involving pictures and video, I think this is right in the scope of O-S-C. If you'r on stage with a O-S-C tablet, being able to monitor the live feed of a camera in the interface to frame it is gold. If you're controlling VJ style shows, being able to click on the thumbnail of a video or feed to send it to display is cool.

Thumbnails of live video to create better control interface is what I have in mind. No processing or anything that does'nt relate UI/control.

[hu-o]

If I try to specify a rtsp url, I got this error. This rtsp URL can be opened and work in VLC.

[jean-emmanuel]

You need to install ffmpeg : http://www.ffmpeg.org/download.html

[jean-emmanuel]

I've been giving this some thoughts and doing a few researchs, and I'm going to remove this from the core app. The proposed test is just a very simple conversion from rtsp stream to base64-encoded jpeg images that are sent to the Image widget. This is easy to handle externally and would avoid O-S-C being responsible for ffmpeg's state (which seems inconsistent). I'll create a repository to demonstrate that.

As for an actual WebRTC implementation, it's a real PITA as far as I can see and is not worth the effort given that the previous solution looks promising enough and matches O-S-C's logic.

[jean-emmanuel]

https://github.com/jean-emmanuel/rtsp2osc

[jean-emmanuel]

@jblanteigne @hu-o
I've been thinking of another solution: adding a "frame" widget that would allow users to embed external http pages to their interface, you'd just have to configure a webrtc stream server that can be reached through http such as https://github.com/mpromonet/webrtc-streamer. What do you think ?

[hu-o]

That would be great, and not only for video preview. Integrating existing web control pages in a OSC project makes sense for some use case where you have equipment that already have web control. Hugo Le mar. 28 août 2018 à 11:32, jean-emmanuel <notifications@github.com> a écrit :

…

@jblanteigne <https://github.com/jblanteigne> @hu-o <https://github.com/hu-o> I've been thinking of another solution: adding a "frame" widget that would allow users to embed external http pages to their interface, you'd just have to configure a webrtc stream server that can be reached through http such as https://github.com/mpromonet/webrtc-streamer. What do you think ? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#259 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/Aj-8tGhLEdYoVKRwFBRdvM-jsknqzfyFks5uVQ5FgaJpZM4S5pdC> .

[jean-emmanuel]

It's in git (widgets/plots/frame). https://github.com/jean-emmanuel/open-stage-control/blob/master/resources/docs/widgets/plots.md#frame

[baronlanteigne]

Good idea. It will be lighter than converting the frames with ffmpeg. I will test whenever I have time. Thanks!

[jean-emmanuel]

@hu-o how did the iframe solution turned out so far for you ?