Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Orange Pi 5+] HDMI IN - Not a video capture device #252

Closed
ggdk123 opened this issue Jul 8, 2023 · 21 comments
Closed

[Orange Pi 5+] HDMI IN - Not a video capture device #252

ggdk123 opened this issue Jul 8, 2023 · 21 comments
Assignees
Labels
bug Something isn't working

Comments

@ggdk123
Copy link

ggdk123 commented Jul 8, 2023

Testing the HDMI In with ffmpeg on ubuntu server. Below is the error message.

ubuntu@ubuntu:~$ sudo v4l2-ctl --list-devices
rk_hdmirx (fdee0000.hdmirx-controller):
        /dev/video0

ubuntu@ubuntu:~$ ffmpeg -i /dev/video0 ~/output.mkv
ffmpeg version 4.4.2-0ubuntu0.22.04.1+rkmpp20230327 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 11 (Ubuntu 11.3.0-1ubuntu1~22.04)
  configuration: --prefix=/usr --extra-version=0ubuntu0.22.04.1+rkmpp20230327 --toolchain=hardened --libdir=/usr/lib/aarch64-linux-gnu --incdir=/usr/include/aarch64-linux-gnu --arch=arm64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-libdrm --enable-rkmpp --enable-version3 --disable-libopenh264 --disable-vaapi --disable-vdpau --disable-decoder=h264_v4l2m2m --disable-decoder=vp8_v4l2m2m --disable-decoder=mpeg2_v4l2m2m --disable-decoder=mpeg4_v4l2m2m --enable-shared
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
[video4linux2,v4l2 @ 0xaaaae42d1930] Not a video capture device.
/dev/video0: No such device
@Joshua-Riek
Copy link
Owner

You need to install some packages for /dev/video0 to appear. Try to install the below packages then reboot your system.

sudo apt-get update
sudo apt-get install -y mali-g610-firmware rockchip-multimedia-config

@Joshua-Riek Joshua-Riek self-assigned this Jul 9, 2023
@Joshua-Riek Joshua-Riek added the bug Something isn't working label Jul 9, 2023
@ggdk123
Copy link
Author

ggdk123 commented Jul 10, 2023

You need to install some packages for /dev/video0 to appear. Try to install the below packages then reboot your system.

sudo apt-get update
sudo apt-get install -y mali-g610-firmware rockchip-multimedia-config

It seems i'm getting the same error after having installed these 2 packages.

@Joshua-Riek
Copy link
Owner

Can you share the output of ls -la /dev/video0?

@ggdk123
Copy link
Author

ggdk123 commented Jul 10, 2023

Can you share the output of ls -la /dev/video0?

ubuntu@ubuntu:~$ ls -la /dev/video0
crw-rw---- 1 root video 81, 0 Jul 10 16:07 /dev/video0

@Joshua-Riek
Copy link
Owner

I think this is an issue with using ffmpeg directly. In my testing, I used gstreamer as shown in the documentation available from orangepi. Below is the example script form orangepi. I think you will need to try with gstreamer.

#!/bin/bash

trap 'onCtrlC' INT
function onCtrlC () {
	echo 'Ctrl+C is captured'
	killall gst-launch-1.0
	exit 0
}

device_id=$(v4l2-ctl --list-devices | grep -A1 hdmirx | grep -v hdmirx | awk -F ' ' '{print $NF}')
v4l2-ctl -d $device_id --set-dv-bt-timings query 2>&1 > /dev/null
width=$(v4l2-ctl -d $device_id --get-dv-timings | grep "Active width" |awk -F ' ' '{print $NF}')
heigh=$(v4l2-ctl -d $device_id --get-dv-timings | grep "Active heigh" |awk -F ' ' '{print $NF}')

es8388_card=$(aplay -l | grep "es8388" | cut -d ':' -f 1 | cut -d ' ' -f 2)
hdmi0_card=$(aplay -l | grep "hdmi0" | cut -d ':' -f 1 | cut -d ' ' -f 2)
hdmi1_card=$(aplay -l | grep "hdmi1" | cut -d ':' -f 1 | cut -d ' ' -f 2)
hdmiin_card=$(arecord -l | grep "hdmiin" | cut -d ":" -f 1 | cut -d ' ' -f 2)

DISPLAY=:0.0 gst-launch-1.0 v4l2src device=${device_id} io-mode=4 ! videoconvert \
	! video/x-raw,format=NV12,width=${width},height=${heigh} \
	! videoscale ! video/x-raw,width=1280,height=720 \
	! autovideosink sync=false 2>&1 > /dev/null &

gst-launch-1.0 alsasrc device=hw:${hdmiin_card},0 ! audioconvert ! audioresample ! queue \
	! tee name=t ! queue ! alsasink device="hw:${hdmi0_card},0" \
	t. ! queue ! alsasink device="hw:${hdmi1_card},0" &

while true
do
	sleep 10
done

@ggdk123
Copy link
Author

ggdk123 commented Jul 11, 2023

tried the gstreamer example script. below is the output, but it doesn't look very good though

ubuntu@ubuntu:~$ ./testhdmiin.sh
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstAudioSrcClock
Redistribute latency...
Redistribute latency...
Redistribute latency...
Caught SIGSEGV
Spinning.  Please run 'gdb gst-launch-1.0 12356' to continue debugging, Ctrl-C to quit, or Ctrl-\ to dump core.
Redistribute latency...
Redistribute latency...
WARNING: from element /GstPipeline:pipeline0/GstAlsaSrc:alsasrc0: Can't record audio fast enough
Additional debug info:
../gst-libs/gst/audio/gstaudiobasesrc.c(847): gst_audio_base_src_create (): /GstPipeline:pipeline0/GstAlsaSrc:alsasrc0:
Dropped 51597 samples. This is most likely because downstream can't keep up and is consuming samples too slowly.
WARNING: from element /GstPipeline:pipeline0/GstAlsaSrc:alsasrc0: Can't record audio fast enough
Additional debug info:
../gst-libs/gst/audio/gstaudiobasesrc.c(847): gst_audio_base_src_create (): /GstPipeline:pipeline0/GstAlsaSrc:alsasrc0:
Dropped 137151 samples. This is most likely because downstream can't keep up and is consuming samples too slowly.
WARNING: from element /GstPipeline:pipeline0/GstAlsaSrc:alsasrc0: Can't record audio fast enough

@ggdk123
Copy link
Author

ggdk123 commented Jul 11, 2023

I think this is an issue with using ffmpeg directly

is this an orange pi 5 specific issue? or all the rk3588 boards are all designed to be so? like Radxa ROCK 5B

@Joshua-Riek
Copy link
Owner

It looks like the above example failed on your end because you are using the server image, I think a desktop environment needs to be started for it to work.

@ggdk123
Copy link
Author

ggdk123 commented Jul 14, 2023

It looks like the above example failed on your end because you are using the server image, I think a desktop environment needs to be started for it to work.

You are right. It works now on the desktop. But latency is very visible, not familiar with gstreamer though. In my view, gstreamer is not exactly equivalent to ffmpeg. ffmpeg is absolutely needed for hdimi in on rk3588 board.

@Joshua-Riek
Copy link
Owner

I agree, but Rockchip's work on ffmpeg has been abandoned. The open-source community must create a fork to add more support and capabilities. There has been some work already done, but it's not something I have looked at yet #246.

@StuartIanNaylor
Copy link

@bagong
Copy link
Sponsor

bagong commented Oct 30, 2023

"Recording" with "ffmpeg -i /dev/video0 ./output.mkv" and "playback" with "mpv /dev/video0" work right away (2023/10/30 (latest of everything, but not yet rl 1.29)), which is way cool! I'd love to use it in OBS or VLC, but seem not to be able to do what can be done with ffmpeg and mpv...

@Joshua-Riek
Copy link
Owner

OBS should work fine to record, but to get OBS to work properly there are a few extra steps. If I recall some env vars need to be set and extra packages. But that's great to hear!

@bagong
Copy link
Sponsor

bagong commented Oct 31, 2023

OBS works fine if you install qt6 platform plugin (hidden in the package qt6-wayland, and yes, qt6, not 5 as everybody assumes ;-) ).

Problem is I see nothing if I define a v4l2 capture device, although OBS does seem to see the hdmiin device:

$: v4l2-ctl --list-devices
rk_hdmirx (fdee0000.hdmirx-controller):
	/dev/video

And this is how OBS sees the device (without producing an image):

obs-opi5plus-hdmi-in-capture-ubuntu22 04

Is there something I can do to understand the problem, solve it?

This is what mpv shows for mpv /dev/video0

hdmiin-opi5plus-mpv-video0

@bagong
Copy link
Sponsor

bagong commented Oct 31, 2023

Vlc gives this output at an attempt to play "video capture device" /dev/video0:

[0000ffff44001150] v4l2 demux error: cannot select input 0: Inappropriate ioctl for device
Using mplane plugin for capture 
[0000ffff44001150] v4l2 demux error: not a radio tuner device
Using mplane plugin for capture 
[0000ffff440033c0] v4l2 stream error: cannot select input 0: Inappropriate ioctl for device

mpv also complained about "Inapropriate ioctl for device" but still produced a video then ;-)

@StuartIanNaylor
Copy link

StuartIanNaylor commented Oct 31, 2023

https://github.com/rigaya/rkmppenc
With rkmppenc --input-format v4l2 -i /dev/video0 \ --input-option channel:0 --input-option ignore_input_error:1 --input-option ts:abs \ --audio-source "hw:<n>:format=alsa/codec=aac;enc_prm=aac_coder=twoloop;bitrate=192" \ -o out.ts

Does it work because they have included --input-option ignore_input_error:1

@bagong
Copy link
Sponsor

bagong commented Nov 2, 2023

It doesn't work for me, the out.ts isn't written (I didn't know how to stop the recording, so I hit ctrl-c, maybe that was wrong). I did however try to identify the term that makes "Inappropriate ioctl for device" go away, and it's not the "ignore_input_error" but "input-option channel:0".

Actually V4L2 Test Bench seems to give working access in a GUI...

@esolk
Copy link

esolk commented Jan 15, 2024

"Recording" with "ffmpeg -i /dev/video0 ./output.mkv" and "playback" with "mpv /dev/video0" work right away (2023/10/30 (latest of everything, but not yet rl 1.29)), which is way cool! I'd love to use it in OBS or VLC, but seem not to be able to do what can be done with ffmpeg and mpv...

I'm running on Ubuntu22.04 and I did 'apt upgrade', but still get Not a video capture device. error while running FFmpeg. MPV shows the same errro. Gstreamer can work with laggy.
How do you make ffmpeg and mpv work?

@StuartIanNaylor
Copy link

Anyone tried the Gstreamer input for Obs instead?
https://github.com/fzwoch/obs-gstreamer

@Retsel023
Copy link

Retsel023 commented Jan 23, 2024

i got my hdmi input to work with ffmpeg. I tweaked it for my nintendo switch. ffmpeg -f v4l2 -video_size 1920x1080 -i /dev/video0 -pix_fmt bgr24 -vf "format=rgb24" -framerate 80 -video_size 1920x1080 -c:v rawvideo -vsync 1 -f sdl "SDL output"

It took me a couple of days to get any input at all. I gave up on OBS gsteamer and the add capture device option. Only black screen and format errors. I might be able to fix the gsstream errors now with the knowledge i have gained but i like the ffmpeg so thats fine. Low latency and decent quality. Note, i do not record my screen i merely wanted to display it on the orangepi and maybe stream it.

For this to work i used the ubuntu 22.04 server image + kde plasma. installed the mali drivers, the ffmpeg dependencies, some libraries and a lot of troubleshooting/debugging.
IMG_20240123_210140

@N0tiK44
Copy link

N0tiK44 commented Mar 18, 2024

i got my hdmi input to work with ffmpeg. I tweaked it for my nintendo switch. ffmpeg -f v4l2 -video_size 1920x1080 -i /dev/video0 -pix_fmt bgr24 -vf "format=rgb24" -framerate 80 -video_size 1920x1080 -c:v rawvideo -vsync 1 -f sdl "SDL output"

It took me a couple of days to get any input at all. I gave up on OBS gsteamer and the add capture device option. Only black screen and format errors. I might be able to fix the gsstream errors now with the knowledge i have gained but i like the ffmpeg so thats fine. Low latency and decent quality. Note, i do not record my screen i merely wanted to display it on the orangepi and maybe stream it.

For this to work i used the ubuntu 22.04 server image + kde plasma. installed the mali drivers, the ffmpeg dependencies, some libraries and a lot of troubleshooting/debugging. IMG_20240123_210140

What were your stats for specifically latency?

and do you have documentation for said necessities?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

7 participants