Looking for validation of config file

[vrv501]

Hi, I'm currently looking to verify if following things can be achieved using SMA

Question 1:
After qBitt has downloaded the video, it will call SMA on the file (Changed settings in qBitt as described in wiki)
SMA has to convert the video(any codec format) to H264 8bit codec (Ignore if it's already in this format) and replace the old file
SMA will finish transcoding and hand back control to qBitt, which will hand back the new transcoded file to *arr

Is this what's happening under the hood?

Question 2:
I want SMA to transcode the video using my custom ffmpeg with h264_v4l2m2m encoder/decoder only (H/W accl is possible using h264_v4l2m2m on raspberry pi 4 for 64bit OS'es)
Example on how SMA should call ffmpeg: ffmpeg -c:v h264_v4l2m2m -b:v 8M -c:a copy -i input.* output.mp4

Does the config file below suit the above behaviour?

pi@rpi4:~/sickbeard_mp4_automator/setup $ cat autoProcess.ini.sample
[Converter]
ffmpeg = /home/pi/bin/ffmpeg
ffprobe = /home/pi/bin/ffprobe
threads = 4
hwaccels = h264_v4l2m2m
hwaccel-decoders = h264_v4l2m2m
hwdevices = h264_v4l2m2m:/dev/video10,h264_v4l2m2m:/dev/video11
hwaccel-output-format = h264_v4l2m2m:h264_v4l2m2m
output-directory =
output-format = mp4
output-extension = mp4
temp-extension =
minimum-size = 0
ignored-extensions = nfo, ds_store
copy-to =
move-to =
delete-original = True
sort-streams = True
process-same-extensions = False
bypass-if-copying-all = False
force-convert = False
post-process = False
wait-post-process = False
detailed-progress = False
opts-separator = ,
preopts =
postopts =
regex-directory-replace = [^\w\-_\. ]


[Permissions]
chmod = 0644
uid = -1
gid = -1

[Metadata]
relocate-moov = True
full-path-guess = True
tag = True
tag-language = eng
download-artwork = poster
sanitize-disposition =
strip-metadata = False
keep-titles = False

[Video]
codec = h264, x264
max-bitrate = 0
bitrate-ratio =
crf = -1
crf-profiles =
preset =
codec-parameters =
dynamic-parameters = False
max-width = 0
profile =
max-level = 0.0
pix-fmt =
prioritize-source-pix-fmt = True
filter =
force-filter = False

[HDR]
codec =
pix-fmt =
space = bt2020nc
transfer = smpte2084
primaries = bt2020
preset =
codec-parameters =
filter =
force-filter = False
profile =

[Audio]
codec = ac3
languages =
default-language =
first-stream-of-language = False
allow-language-relax = True
channel-bitrate = 128
variable-bitrate = 0
max-bitrate = 0
max-channels = 0
prefer-more-channels = True
default-more-channels = True
profile =
filter =
force-filter = False
sample-rates =
sample-format =
copy-original = False
copy-original-before = False
aac-adtstoasc = False
ignore-truehd = mp4, m4v
ignored-dispositions =
unique-dispositions = False

[Universal Audio]
codec = aac
channel-bitrate = 128
variable-bitrate = 0
first-stream-only = False
move-after = False
profile =
filter =
force-filter = False

[Audio.ChannelFilters]
6-2 = pan=stereo|FL=0.5*FC+0.707*FL+0.707*BL+0.5*LFE|FR=0.5*FC+0.707*FR+0.707*BR+0.5*LFE

[Subtitle]
codec = mov_text
codec-image-based =
languages =
default-language =
first-stream-of-language = False
encoding =
burn-subtitles = False
burn-dispositions =
embed-subs = True
embed-image-subs = False
embed-only-internal-subs = False
filename-dispositions = forced
ignore-embedded-subs = False
ignored-dispositions =
unique-dispositions = False
attachment-codec =

h264_v4l2m2m supports both encoding & decoding

pi@rpi4:~ $ ffmpeg -codecs | grep -i h264
ffmpeg version N-105691-gc439c6b191 Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 10 (Debian 10.2.1-6)
  configuration: --prefix=/home/pi/ffmpeg_build --pkg-config-flags=--static --extra-cflags=-I/home/pi/ffmpeg_build/include --extra-ldflags=-L/home/pi/ffmpeg_build/lib --extra-libs='-lpthread -lm' --ld=g++ --bindir=/home/pi/bin --arch=aarch64 --enable-gpl --enable-gnutls --enable-libaom --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libwebp --enable-libdrm --enable-libsvtav1 --enable-libdav1d --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libxml2 --enable-nonfree
  libavutil      57. 21.100 / 57. 21.100
  libavcodec     59. 21.100 / 59. 21.100
  libavformat    59. 17.102 / 59. 17.102
  libavdevice    59.  5.100 / 59.  5.100
  libavfilter     8. 27.100 /  8. 27.100
  libswscale      6.  5.100 /  6.  5.100
  libswresample   4.  4.100 /  4.  4.100
  libpostproc    56.  4.100 / 56.  4.100

Codecs:
 D..... = Decoding supported
 .E.... = Encoding supported
 ..V... = Video codec
 ..A... = Audio codec
 ..S... = Subtitle codec
 ...I.. = Intra frame-only codec
 ....L. = Lossy compression
 .....S = Lossless compression
---------
 DEV.LS h264                 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (decoders: h264 h264_v4l2m2m ) (encoders: libx264 libx264rgb h264_v4l2m2m h264_vaapi )

[mdhiggins]

h264_v4l2m2m not supported at the moment but easy to add. Documentation on this encoder is sparse, if you have any post it and I'll take a look

I don't have a pi to test on, can you run also run ffmpeg -hwaccels and post that too so I can take a look and I'll make some quick updates and get back to you

Also since apparently there's a hevc variant as well can you run ffmpeg -codecs | grep -i hevc and I can do them both maybe

[vrv501]

Article used for enabling this encoder/decoder: h264_v4l2m2m
Different h264 encoders/decoders: SO

Using the h264_v4l2m2m the Pi can encode 1080p video at 53-60 FPS, compared to just 8-10 FPS when using the libx264 CPU decoder.

Also to add looks like h264_v4l2m2m requires this argument -vf format=yuv420p and .mp4 output container: Jellyfin Issues jellyfin/jellyfin-ffmpeg#310
SO Post for yuv420p arg: SO

pi@rpi4:~ $ ffmpeg -hwaccels
ffmpeg version N-105691-gc439c6b191 Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 10 (Debian 10.2.1-6)
  configuration: --prefix=/home/pi/ffmpeg_build --pkg-config-flags=--static --extra-cflags=-I/home/pi/ffmpeg_build/include --extra-ldflags=-L/home/pi/ffmpeg_build/lib --extra-libs='-lpthread -lm' --ld=g++ --bindir=/home/pi/bin --arch=aarch64 --enable-gpl --enable-gnutls --enable-libaom --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libwebp --enable-libdrm --enable-libsvtav1 --enable-libdav1d --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libxml2 --enable-nonfree
  libavutil      57. 21.100 / 57. 21.100
  libavcodec     59. 21.100 / 59. 21.100
  libavformat    59. 17.102 / 59. 17.102
  libavdevice    59.  5.100 / 59.  5.100
  libavfilter     8. 27.100 /  8. 27.100
  libswscale      6.  5.100 /  6.  5.100
  libswresample   4.  4.100 /  4.  4.100
  libpostproc    56.  4.100 / 56.  4.100
Hardware acceleration methods:
vdpau
vaapi
drm

pi@rpi4:~ $ ffmpeg -codecs | grep -i hevc
ffmpeg version N-105691-gc439c6b191 Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 10 (Debian 10.2.1-6)
  configuration: --prefix=/home/pi/ffmpeg_build --pkg-config-flags=--static --extra-cflags=-I/home/pi/ffmpeg_build/include --extra-ldflags=-L/home/pi/ffmpeg_build/lib --extra-libs='-lpthread -lm' --ld=g++ --bindir=/home/pi/bin --arch=aarch64 --enable-gpl --enable-gnutls --enable-libaom --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libwebp --enable-libdrm --enable-libsvtav1 --enable-libdav1d --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libxml2 --enable-nonfree
  libavutil      57. 21.100 / 57. 21.100
  libavcodec     59. 21.100 / 59. 21.100
  libavformat    59. 17.102 / 59. 17.102
  libavdevice    59.  5.100 / 59.  5.100
  libavfilter     8. 27.100 /  8. 27.100
  libswscale      6.  5.100 /  6.  5.100
  libswresample   4.  4.100 /  4.  4.100
  libpostproc    56.  4.100 / 56.  4.100
  
Codecs:
 D..... = Decoding supported
 .E.... = Encoding supported
 ..V... = Video codec
 ..A... = Audio codec
 ..S... = Subtitle codec
 ...I.. = Intra frame-only codec
 ....L. = Lossy compression
 .....S = Lossless compression
---------
 DEV.L. hevc                 H.265 / HEVC (High Efficiency Video Coding) (decoders: hevc hevc_v4l2m2m ) (encoders: libx265 hevc_v4l2m2m hevc_vaapi )

[vrv501]

@mdhiggins Let me know if you need any other info from raspberry pi

[mdhiggins]

I was looking for more technical documentation similar to what's available for other hardware encoders like vaapi, QSV, or nvenc. Doesn't seem to be much out there beyond some very basic tutorials.

The sample you provided seems to only be a decoder example since you're specifying the codec before the input file
ffmpeg -c:v h264_v4l2m2m -b:v 8M -c:a copy -i input.* output.mp4

The example in the article is using the encoder since it comes after

ffmpeg -i video.mp4 -c:v h264_v4l2m2m -b:v 8M -c:a copy test.mp4

One thing I did notice from reviewing some of the source code and updates for this hwaccel option is that it behaves a little differently in that it doesn't report itself on the -hwaccels list so I made some updates so that you can still use the decoder in this case

2ced3be

Its probably unlikely that yuv420p is the only pix_fmt the encoder can use, you can check by running

ffmpeg -h encoder=h264_v4l2m2m

Look for the line

Supported pixel formats:

As for settings you'll need, here are the important ones

[Converter]
hwaccels = 
hwaccel-decoders = h264_v4l2m2m
hwdevices = 
hwaccel-output-format = 

[Video]
codec = h264_v4l2m2m

Depending on supported pix_fmt options you may need to adjust that setting as well

[mdhiggins]

You also may be able to combine this with the drm hwaccel format to avoid any software steps but I can't confirm this without a test device and it does get more complicated

[mdhiggins]

Closing this out since support has been added, continue discussion if there are any issues here and I can reopen

[vrv501]

@mdhiggins Looks like h264_v4l2m2m doesn't list it's supported pix_fmt. However when trying to encode, it throws error without this argument.
[h264_v4l2m2m @ 0x5587de52e0] Encoder requires yuv420p pixel format.

pi@rpi4:~ $ ffmpeg -hide_banner -h encoder=h264_v4l2m2m
Encoder h264_v4l2m2m [V4L2 mem2mem H.264 encoder wrapper]:
    General capabilities: delay hardware
    Threading capabilities: none
h264_v4l2m2m_encoder AVOptions:
  -num_output_buffers <int>        E..V....... Number of buffers in the output context (from 6 to INT_MAX) (default 16)
  -num_capture_buffers <int>        E..V....... Number of buffers in the capture context (from 4 to INT_MAX) (default 4)

[mdhiggins]

Got it, this can technically be solved by setting the pix-fmt option in autoProcess but I went ahead and just hard-coded it for the h264_v4l2m2m class

37c9bfa

[vrv501]

Thanks man! I shall setup SMA in my Pi right away!

[nov1n]

@vrv501 I'm trying to achieve the same thing on my Raspberry Pi 4B (64 bit). I know little about ffmpeg so I wondered if you could share your approach? I adopted the suggested config changes:

[Converter]
hwaccels =
hwaccel-decoders = h264_v4l2m2m
hwdevices =
hwaccel-output-format =

[Video]
codec = h264_v4l2m2m

And this is my docker-compose for reference:

 radarr:
    image: mdhiggins/radarr-sma
    container_name: radarr
    restart: unless-stopped
    volumes:
      - /home/pi/.config/radarr/config:/config
      - /home/pi/.config/sma/config:/usr/local/sma/config
      - /home/pi/data:/data
    environment:
      - PUID=13002
      - PGID=13000
      - UMASK=002
      - SMA_HWACCEL=true

Logs:

2023-06-17 12:36:06 - resources.mediaprocessor - DEBUG - Selected hwaccel options:
2023-06-17 12:36:06 - resources.mediaprocessor - DEBUG - []
2023-06-17 12:36:06 - resources.mediaprocessor - DEBUG - Selected hwaccel decoder pairs:
2023-06-17 12:36:06 - resources.mediaprocessor - DEBUG - ['h264_v4l2m2m']
2023-06-17 12:36:06 - resources.mediaprocessor - DEBUG - FFMPEG hwaccels:
2023-06-17 12:36:06 - resources.mediaprocessor - DEBUG - ['vdpau']
2023-06-17 12:36:06 - resources.mediaprocessor - DEBUG - Input format yuv420p10le bit depth 10.
2023-06-17 12:36:06 - resources.mediaprocessor - DEBUG - canBypassConvert returned False.
...
/usr/local/bin/ffmpeg -i "/data/media/movies/xxx.mkv" -vcodec libx264 ...

I would expect -vcodec libx264 to be h264_v4l2m2m as specified in the config file. I do see h264_v4l2m2m is mentioned, but my CPU is still maxing out, so I'm not sure what to make of it.

Thanks in advance :)

[mdhiggins]

Could you share the full logs? I'll take a look

[nov1n]

Sure, sent you an email.