Hardware acceleration for video transcoding

[ghost]

I'm running PeerTube on a server with a GPU that I was hoping to use for transcode acceleration. In my particular case I'd really be interested in nvenc and cuvid.

Honestly I was hoping to see some way to generically customize the ffmpeg parameters for each transcode level; maybe that sort of configuration functionality could allow admins to tweak video quality and bandwidth settings as well as supporting platform-specific codecs?

I'd love some guidance on how other folks would like to see this support implemented, but I'm happy to do the work.

[Jorropo]

Honestly I was hoping to see some way to generically customize the ffmpeg parameters for each transcode level; maybe that sort of configuration functionality could allow admins to tweak video quality and bandwidth settings as well as supporting platform-specific codecs?

That isn't a so good idea because the player require a very specific encoding, bandwidth, ... to allow scrolling in the video. This is why this isn't implemented :)
Hardware accels can be simply add, just require to add a parameters to config.

[ghost]

That isn't a so good idea because the player require a very specific encoding, bandwidth, ... to allow scrolling in the video. This is why this isn't implemented :)

Ah, that makes sense! I was surprised not to at least see a way to control bit rate or overall encoding quality though. On my system at least, the ffmpeg command line doesn't include any rate or quality info, and I was hoping for a way to tune that or at least see what the current bandwidth results form.

(sample ffmpeg command line I'm seeing now...)

/usr/local/bin/ffmpeg -i /var/www/peertube/storage/videos/a3fba5cd-fd18-4db3-ab4a-c80849d29c93-1080.mp4 -y -vcodec libx264 -r 60 -threads 4 -movflags faststart -b_strategy 1 -bf 16 /var/www/peertube/storage/videos/3-transcoded.mp4

When you say hardware accels can be simply added, do you mean there's already way to do this, or are you saying it could be a simple new feature to implement?

Thanks again!

-micah

[rigelk]

@scanlime @Jorropo citing FFmpeg's guide on hardware acceleration:

External wrapper decoders are used by setting a specific decoder with the -codec:v option. Typically they are named codec_api (for example: h264_cuvid). These decoders require the codec to be known in advance, and do not support any fallback to software if the stream is not supported.

Which means we would have to detect support for each and every hardware accelerator ; and failure to do so would not have a fallback to software, which is a bit problematic. However, we already have an example of a detection function for simple h264 though, so this could be done for hardware accelerators I guess. We would just have to list them.

Just note:

Hardware encoders typically generate output of significantly lower quality than good software encoders like x264, but are generally faster and do not use much CPU resource.

It's fine by me since we care more about a faster encoding than a bit-perfect encoding.

---

As for profiles/presets, we already have some in the code:

PeerTube/server/helpers/ffmpeg-utils.ts

Lines 178 to 210 in 65fcf1e

	/**
	* A slightly customised version of the 'veryfast' x264 preset
	*
	* The veryfast preset is right in the sweet spot of performance
	* and quality. Superfast and ultrafast will give you better
	* performance, but then quality is noticeably worse.
	*/
	function veryfast (_ffmpeg) {
	_ffmpeg
	.preset(standard)
	.outputOption('-preset:v veryfast')
	.outputOption(['--aq-mode=2', '--aq-strength=1.3'])
	/*
	MAIN reference: https://slhck.info/video/2017/03/01/rate-control.html
	Our target situation is closer to a livestream than a stream,
	since we want to reduce as much a possible the encoding burden,
	altough not to the point of a livestream where there is a hard
	constraint on the frames per second to be encoded.
	why '--aq-mode=2 --aq-strength=1.3' instead of '-profile:v main'?
	Make up for most of the loss of grain and macroblocking
	with less computing power.
	*/
	}
	/**
	* A preset optimised for a stillimage audio video
	*/
	function audio (_ffmpeg) {
	_ffmpeg
	.preset(veryfast)
	.outputOption('-tune stillimage')
	}

There just isn't a way to select them when you upload your video for the moment. So it's not that we cannot let users/admins customize their encoding profiles, it's more that we have a baseline of keeping a few things for maximum compatibility: the faststart flag, the h264 video codec, the aac audio codec. Now this will probably change, but not in the near future.

[ghost]

Thanks again, Good point about the potential quality trade off. I’ll have to try manually re-transcoding some of these videos just to convince myself the quality is worthwhile still. Even with software encoding though, hardware decode seems like it could help a lot, especially when the source video is high bitrate.

…

On Tue, Aug 21, 2018 at 4:23 AM Rigel Kent ***@***.***> wrote: @scanlime <https://github.com/scanlime> @Jorropo <https://github.com/Jorropo> citing FFmpeg's guide on hardware acceleration <https://trac.ffmpeg.org/wiki/HWAccelIntro>: External wrapper decoders are used by setting a specific decoder with the -codec:v option. Typically they are named codec_api (for example: h264_cuvid). These decoders require the codec to be known in advance, and do not support any fallback to software if the stream is not supported. Which means we would have to detect support for each and every hardware accelerator ; and failure to do so would not have a *fallback to software*, which is a bit problematic. However, we already have an example of a detection function for simple h264 though, so this could be done for hardware accelerators I guess. We would just have to list them. Just note: Hardware encoders typically generate output of significantly lower quality than good software encoders like x264, but are generally faster and do not use much CPU resource. It's fine by me since we care more about a faster encoding than a bit-perfect encoding. ------------------------------ As for profiles/presets, we already have some in the code: https://github.com/Chocobozzz/PeerTube/blob/65fcf1e9c28cd801c3c7b0c70f28d1ed52d3fd3a/server/helpers/ffmpeg-utils.ts#L178-L210 There just isn't a way to select them when you upload your video for the moment. So it's not that we cannot let users/admins customize their encoding profiles, it's more that we have a baseline of keeping a few things for maximum compatibility: the faststart flag, the h264 video codec, the aac audio codec. Now this will probably change, but not in the near future. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#938 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA4YpApByu6nhCgIAcgsTBI1-CYSDE69ks5uS-27gaJpZM4WFey1> .

[karibuTW]

Hello guys,
My server is taking quite some time to transcode some large videos and GPU acceleration would be super good to see!
Any way to implement this?

Thank you

[Jorropo]

Hello guys,
My server is taking quite some time to transcode some large videos and GPU acceleration would be super good to see!
Any way to implement this?

Thank you

Il have been a while since I looked in the souce code but from what I remember ffmpeg already have gpu acceleration enabled.

[karibuTW]

I see, I guess my system is not ready neither. I need to install NVENC.
Let me give it a try!

[Jorropo]

@karibuTW it's also that ffmpeg is CPU focused, don't except any huge improvement by GPU acceleration.
If you would use 100% of your GPU an OpenCL or Vulkan CL transcoder would be better (but that would require some relatively beaffy GPUs wich the average server doesn't have).

[karibuTW]

I see, I have only a G710 in the server, so i don't expect big speed neither...It might may be even be counterproductive in this case. I need to find another way.

My Xeon E3 1220v5 (4C/4T) @3 GHz is taking ages to encode... (I've asked only 360p and 720p) and have configured it with 4 threads.
I just realized that FFMPEG is taking 150% of CPU but probably the nice is too gentle. I have some CPU intensive application (BOINC) to leverage the "unused" cycle with a nice of 19. If I stop BOINC, FFMPEG goes to 350% (Not yet 400%, but better!).
May be I need to adjust the nice (15 now)

EDIT: Well, after changing the nice of the process, it is not much better. Anyway, I let FFMPEG to run at the moment and will see.
I forgot it was that intensive ;) The CPU might not be appropriate.

[Jorropo]

May be I need to adjust the nice (15 now)

@karibuTW
Idealy you would like to have FFMPEG running at a lower nice than boinc, so it's boinc waiting for ffmpeg not the inverse.
Also depending of the boinc project it may have huge time beetween releasing to the kernel (if it does lots of computation it can happend seconds beetween syscalls), reducing opportunities of a context switch.
I know there is an option to only run when the cpu is only used bellow a certain threashold in boinc. If you do that when FFMPEG is running boinc will pause it self and run latter.
(Also nice shouldn't really care about the difference beetween 2 process, just if that lower or higher (if you have boinc at 19 and ffmpeg at 5 it should be the same as boinc as 19 and ffmpeg as 15), but I've seen some case where this where actualy important)

[karibuTW]

Make sense Jorropo, thanks for all those sharing!

[ghost]

@karibuTW it's also that ffmpeg is CPU focused, don't except any huge improvement by GPU acceleration.
If you would use 100% of your GPU an OpenCL or Vulkan CL transcoder would be better (but that would require some relatively beaffy GPUs wich the average server doesn't have).

This is not true, video encoding does not require a particularly beefy GPU. Plenty of older mid-range nvidia chips can encode multiple HD streams in real time.

My understanding is that it would be relatively easy to suppprt hardware accel at this point (it's just a few ffmpeg options) but this involves a quality tradeoff and it might be more acceptable to do for specific formats than as a general default.

[Jorropo]

This is not true, video encoding does not require a particularly beefy GPU. Plenty of older mid-range nvidia chips can encode multiple HD streams in real time.

@scanlime
I know that (that why I've put "relatively"), older mid-range nvidia chips are totaly adequate but relatively to what VPS usualy gets (nothing or an integrated intel/amd GPU) this is a lot.

[ghost]

fwiw even servers without a discrete GPU often have some hardware video encoding built into the CPU (intel quick sync) which ffmpeg does support

[ghost]

it's also maybe worth mentioning that both hardware decode and harware encode support could be advantageous, and hardware decoding doesn't change the output quality at all so it's "free" to implement on any environment that has the hardware and library support

[Avatat]

I would like to put in my two cents :)
As I understand, by default, ffmpeg-utils.ts uses libx264 codec to encode video. If I'm right, all encoding (and decoding too) will happen on a CPU: https://trac.ffmpeg.org/wiki/Encode/H.264
Of course, CPU transcoding is very slow and heavy.

I think that hardware acceleration is a "must-have" for real-use PeerTube instances.

Hardware acceleration needs three things:

1. Hardware capable of de/encoding
2. Drivers and libraries which will support used hardware
3. FFmpeg compiled with hardware-accelerated codecs, and use of these codecs in the command line

At this moment, I have a small experience with NVIDIA's accelerated transcoding, so I can speak only about it.
To use NVIDIA's acceleration, you have to choose a supported GPU. For my experiment I have chosen an old K2200 GPU, because it was cheap, it has full support for H.264 codec, and supports an unlimited number of concurrent de/encoding sessions.
NVIDIA K2200 is capable to transcode 1080p30 (H.264) movie to 720p, 540p, and 480p (H.264) simultaneously at almost 12x speed. For example, it will transcode 60 minutes long movie in 5 minutes, to three different resolutions. And it uses almost no CPU when transcoding!
I have to notice, that GTX/RTX GPUs have limited encoding capabilities to 3 concurrent sessions, but some hacks exist to omit this limit.

In the future, I plan to learn how to do Intel and AMD acceleration - then I could write more about it.

I would like to add or help in adding a hardware-accelerated transcoding feature to PeerTube.

[ghost]

I hope you mean "must support" rather than "must have"... I too am enthusiastic about hardware encoding but there are some big limitations to keep in mind:

• Lots of people are running on servers without GPUs
• Hardware transcodes may be lower quality than libx264
• Hardware transcoders may have bugs that need to be worked around for specific formats/videos

Ideally we'd have some kind of pluggable support for transcoders running elsewhere; until then, it seems necessary to give admins a lot of control over when hardware transcoding happens and what settings it uses.

[Avatat]

Definitely - we have to implement it in a pluggable and fully configurable way.

[Avatat]

I had made some Intel-based transcoding and created a new page with the results: https://github.com/Avatat/sandbox/blob/master/tests/ffmpeg_transcoding.md

[Chocobozzz]

In the next release, you'll be able to create plugins to change transcoding options/encoders (and so use an encoder that supports hardware acceleration): https://docs.joinpeertube.org/contribute-plugins?id=add-new-transcoding-profiles

[snakeeater4526]

hi, sorry to pick that old issue but.... i try to make work VAAPI in peertube for transcoding.

the vaapi plugin give me input/output error.

and the transcoding-profile-debug , i tried that:

{
"vod": [
{
"encoderName": "aac",
"profileName": "test",
"outputOptions": []
},
{
"encoderName": "h264_vaapi",
"profileName": "test",
"outputOptions": []
}
],

"live": []
}

and it give me error 1 : conversion failed....

If anyone have an idea.... i would be really happy to hear it.... i'm not someone that like asking help but here i searched everywhere on google and i'm working on that particular project since non stop 48h

THANK YOU

[Avatat]

I would start from running plain ffmpeg using h264_vaapi encoder - I believe it will fail too.