send video file with audio

[voidless-03]

Has anyone been able to send a video file (mp4) using ndi?

[WalkerKnapp]

To do that with a library like this, you'll have to decode the video file with something like ffmpeg. I like Bytedeco's ffmpeg wrapper. It's a little bit messy, but I threw together a quick demo that does this:

https://gist.github.com/WalkerKnapp/298690a38d69b9af902785b53e9d48ce

This depends on

implementation group: 'org.bytedeco', name: 'ffmpeg-platform', version: '4.2.1-1.5.2'
implementation 'me.walkerknapp:devolay:2.0.1'

This will have some issues with strange video formats, but it seemed to be working well when I tested it. Please let me know if it works, or if you can get something written like this!

[voidless-03]

Hey thanks for the reply! The video works with the ndi output but the audio seems to stutter quite a bit. I'm using an mp4 video format.

[WalkerKnapp]

Could you email me an example video at walker(at)walkerknapp.me? I think that issue is probably specific to the audio sample format.

[voidless-03]

Sent! It seems that with some videos the audio lags behind and isn't synced like the original video.

[WalkerKnapp]

I don't think this is the full issue, but something to try is to switch line 131 from new DevolaySender("Send Video Example", null, false, true); to new DevolaySender("Send Video Example", null, true, false); to clock to the video frames instead of audio frames. I updated the gist with that change.

[voidless-03]

Works fine now! Another thing I noticed is that videos with a resolution of 1080p also stutter with the same format as the other video

[WalkerKnapp]

At that point the issue may be that the decoding/demuxing is unbuffered, I'd imagine it's a latency issue from reading off of the disk, and that is too much of a delay when everything is happening on one thread. I'm working on a more proper version, but it'll be a bit more complicated.

[WalkerKnapp]

I just published that better version at https://github.com/WalkerKnapp/video2ndi. This properly multithreads the demuxing, audio decoding, and video decoding, and buffers some incoming packets so they don't have to be read off the disk. It could maybe be improved more with a more advanced buffering scheme, maybe after decoding the frames but before sending them to NDI.

Right now, it is 100% smooth on my computer playing videos at 1080p60fps and 4k30fps, but struggles with 4k60fps. It is probably doable if that data rate is required, but it would require a lot of optimization, and at that point it would be better to rewrite this in C to avoid Java overhead.

Please let me know how this works for you!

[voidless-03]

Hey thanks! Ill try it out!

[WalkerKnapp]

This issue seems finished to me, but feel free to reopen if you need any more help!