obs-ffmpeg: Add texture-based hardware AMD encoder

[jp9000]

Description

Add support for texture-based AMD encoding, with both H264, HEVC, and HDR support. Fall back to FFmpeg when texture-based encoding cannot be used for whatever reason.

Motivation and Context

The original implementation of the AMD hardware encoder is a submodule that's old, poorly maintained, a bit complicated and difficult to maintain, and should probably be considered deprecated at this point.

This PR instead implements the AMD hardware encoder in a similar approach to jim-nvenc: a very minimalistic texture-based implementation that falls back to FFmpeg if texture-based encoding is not available for whatever reason. With this sort of implementation, it makes the AMD encoder easier to maintain and focuses specifically on texture-based encoding, which should be OBS' strength.

I based this code upon the work done by AMD/Luxoft themselves in #4538, which already was a simplified texture-based implementation, but I further simplified their implementation according to our preferences/tastes. In addition, I also added support for P010 textures for HEVC, updated the API to the latest release, included support for HDR, added a subprocess to check for AMD encoder support called on startup, and implemented an FFmpeg fallback for when texture-based encoding is unavailable.

I wanted people to be able to pass custom options, but I didn't want to implement a complicated properties system, so instead I opted for the ability to pass custom options via text, similar to how the x264 encoder works; except in this case, we use the FFmpeg option names so the options can safely be used with the FFmpeg fallback. (We should probably do something similar for jim-nvenc at some point).

It'd be preferable to deprecate and eventually fully remove the old AMD hardware implementation in favor of this.

Please leave comments and testing results if anyone has a chance to test this. I have only tested this on two machines.

How Has This Been Tested?

Tested H264, HEVC, and HDR. Everything appears to be functioning, although I have probably not tested every option or command line param option. This probably needs more testing on various machines as well. To be able to test HEVC, you must build it yourself and turn on the ENABLE_HEVC CMake option when configuring.

Again, if anyone else could test this with their AMD video cards, the help would be greatly appreciated.

Types of changes

• New feature (non-breaking change which adds functionality)
• Performance enhancement (non-breaking change which improves efficiency)
• Code cleanup (non-breaking change which makes code smaller or more readable)

Checklist:

• My code has been run through clang-format.
• I have read the contributing document.
• My code is not on the master branch.
• The code has been tested.
• All commit messages are properly formatted and commits squashed where appropriate.
• I have included updates to all appropriate documentation.

[jp9000]

Whoops, seems I'll have to fix building on non-windows. I admit I didn't try compiling this branch on other operating systems.

[WizardCM]

Testers: Please mention the model of card + driver combination tested, as that has usually been the necessary information to help debug the old encoder, and provide a log file. This'll help us ensure everyone gets the best experience. :)

[tytan652]

This should fix building on non-Windows system.

[tytan652]

We could check for the AMF lib by using os_dlopen(AMF_DLL_NAMEA) before registering FFmpeg AMF encoders.

AMF_DLL_NAMEA comes from core/Factory.h.

And also check for available device.

[flaeri]

AMD Ryzen 5 4500U APU
Driver: 22.5.1

Been testing a bit, and figured I would ask about a couple of items.

FFmpeg options.
I feel like whatever I put into this box gets ignored. So far I've tried profile=main and level=4.0 and level=40.
None of these seem to generate a file following this direction. Both MediaInfo and ffprobe both claim that there is no change, and seems to always stay at Profile: High, Level: 4.2

Additionally, I think it's a little bit unfortunate that what I put into this field does not get printed to the log. As someone who does support quite frequently, I know users love putting stuff into those boxes, and I would quite like to know what (could be crucial info).

Something that fits into this, is that there dosent seem to be any feedback in the log about the state of the parameters set. I don't know if I've made typo'd an input, or if what I've put in there is valid. Nothing gets noted in the log about the state of the params, and the encoder simply ignores it (unsure if this is because it might ignore everything at the moment). For instance, putting in noexist=yes.

---

There are some errors in my log, but that might just because my poor APU lacks support (it dosent have b-frames for instance), so it wont surprise me if it lacks the Pre-Analysis feature as well. Unsure about the AMF_INVALID_ARG (not related to any params/args I specify at least).

16:09:58.245: [texture-amf: 'advanced_video_recording'] Failed to set property 'EnablePreAnalysis': AMF_FAIL
16:09:58.249: [texture-amf: 'advanced_video_recording'] Failed to set property 'TIMEOUT': AMF_INVALID_ARG

Other than that, seems to work well. Generates valid recordings, streams and does what one would expect 👍

Logs:
jim1.txt

Edit1: Forgot to include driver version

[cs9kc]

I am getting similar errors and behavior as @flaeri is. None of my user-input settings are obeyed (I cross-checked with texture-amf-opts.hpp to ensure I was using the right settings that it was looking for). I could still encode video, however, and like flaeri's experience, it looked fine.

"Reference" RX 6800, 22.5.1.

I also had this set of persistent errors regardless of what I did:

19:02:11.845: [AMF] 2022-05-18 19:02:11.844      788 [AMFEncoderCoreH264]   Error: ..\..\..\..\..\runtime\src\components\EncoderCore\EncoderCoreH264Impl.cpp(1165):AMF_ERROR 1 : AMF_FAIL: Failed to set-up PA module
19:02:11.845: [texture-amf: 'advanced_video_stream'] Failed to set property 'EnablePreAnalysis': AMF_FAIL
19:02:11.845: [AMF] 2022-05-18 19:02:11.845      788 [AMFEncoderCoreH264] Warning: SetProperty TIMEOUT not found
19:02:11.845: [texture-amf: 'advanced_video_stream'] Failed to set property 'TIMEOUT': AMF_INVALID_ARG```

[jp9000]

FFmpeg options [not working]

I'll check them out and see if I can reproduce.

I also had this set of persistent errors regardless of what I did

I think some of those messages may either be due to: having older drivers, or having an older device. I'd assume it's more of a driver thing, but I'm not sure. Are you on the latest drivers?

Technically those error messages doesn't stop the encoder from working. They're just messages saying that it tried to set a property but that property didn't work or doesn't exist.

[cs9kc]

jp9000,

Thank you for checking on the inputs for the new AMD encoder! All of the work you have done is very much appreciated!

Regarding @flaeri's statement:
Additionally, I think it's a little bit unfortunate that what I put into this field does not get printed to the log. As someone who does support quite frequently, I know users love putting stuff into those boxes, and I would quite like to know what (could be crucial info).
I second this.

I appreciate your use of "typical" ffmpeg commands for things like bitrate, buffer, etc, but I have two questions:

• Will commands not currently in ffmpeg's AMF implementation be added to OBS Studio?
• Will traditional ffmpeg commands that aren't cucrrently listed in https://github.com/obsproject/obs-studio/blob/obs-amf/plugins/obs-ffmpeg/texture-amf-opts.hpp, still be passed to the texture-based encoder properly?

For example, texture-amf-opts.hpp lists the AMF-included b-frame settings, but would the -"bf [number]" (or -v:bf [number]) command from ffmpeg work to set the number of B-frames used in the AMF texture-based encoder (AMF_VIDEO_ENCODER_B_PIC_PATTERN; ), (I understand formatting will be "bf=#")? Likewise missing are buffer size settings, QVBR, and QVBR quality level for H264/AVC.

On the other hand, several AMF-specific features that weren't included in ffmpeg aren't listed in texture-amf-opts.hpp. Notably missing is preanalysis(which we want), pre-encoding(which we don't want), adaptive miniGOP, among others.

Thanks!

[Andy76swe]

Where can one find the compiled version so can test it out with my 6900XT?

[tytan652]

Where can one find the compiled version so can test it out with my 6900XT?

Until Jim re-push, it's there:
https://github.com/obsproject/obs-studio/actions/runs/2367858181
In the artifacts section at the bottom of the page.

Edit: link updated

[jp9000]

Apparently AMD plans on removing preanalysis because AMD marked it as deprecated.

[jp9000]

Okay, I've fixed the params, they should work now. I was calling them after initialization rather than before, which would cause them to do nothing. I also added some logging.

I still haven't added preanalysis though, the reason why is because it throws deprecation warnings, so I'm still a bit on the fence about it.

[cs9kc]

Okay, I've fixed the params, they should work now. I was calling them after initialization rather than before, which would cause them to do nothing. I also added some logging.

Thanks! Will try it out when I get some time in the coming days!

I still haven't added preanalysis though, the reason why is because it throws deprecation warnings, so I'm still a bit on the fence about it.

Ah, you may be calling "AMF_VIDEO_ENCODER_RATE_CONTROL_PREANALYSIS_ENABLE"

Below are lines 162-164 of VideoEncoderVCE.h from AMF: https://github.com/GPUOpen-LibrariesAndSDKs/AMF/blob/master/amf/public/include/components/VideoEncoderVCE.h

#define AMF_VIDEO_ENCODER_PRE_ANALYSIS_ENABLE L"EnablePreAnalysis" // bool; default = false; enables the pre-analysis module. Currently only works in AMF_VIDEO_ENCODER_RATE_CONTROL_METHOD_PEAK_CONSTRAINED_VBR mode. Refer to AMF Video PreAnalysis API reference for more details.

#define AMF_VIDEO_ENCODER_PREENCODE_ENABLE L"RateControlPreanalysisEnable" // amf_int64(AMF_VIDEO_ENCODER_PREENCODE_MODE_ENUM); default = AMF_VIDEO_ENCODER_PREENCODE_DISABLED; enables pre-encode assisted rate control

#define AMF_VIDEO_ENCODER_RATE_CONTROL_PREANALYSIS_ENABLE L"RateControlPreanalysisEnable" // amf_int64(AMF_VIDEO_ENCODER_PREENCODE_MODE_ENUM); default = AMF_VIDEO_ENCODER_PREENCODE_DISABLED; enables pre-encode assisted rate control. Deprecated, please use AMF_VIDEO_ENCODER_PREENCODE_ENABLE instead.

AMD had to go make it confusing by calling Pre-Encode "preanalysis"... until they added another feature later on called... wait for it... "PreAnalysis"

Ugh, haha. The 'new' PreAnalysis from line 162 above is what I'm after. Pre-Encode disables B-frames and other things that are also desiserable, so I don't have much interest in Pre-Encode.

[jp9000]

Oh so they renamed it then. I can add it back if that's the case.

[cs9kc]

Hello again! Passing options does indeed work as expected when using the options found in https://github.com/obsproject/obs-studio/blob/obs-amf/plugins/obs-ffmpeg/texture-amf-opts.hpp

I looked through your code, but could not find out if it was possible to pass options to the encoder, that aren't found in texture_amf_options.hpp. If that is not possible yet, that is fine, though I would like to request that a few more feature flags be added to AVC if time is available to do so.

From https://github.com/GPUOpen-LibrariesAndSDKs/AMF/blob/master/amf/doc/AMF_Video_Encode_API.pdf. All of the following begin with "AMF_VIDEO_ENCODER", so it is omitted below:

• ENCODER_TARGET_BITRATE
• ENCODER_PEAK_BITRATE
• Add QVBR to RATE_CONTROL_METHOD
• QVBR_QUALITY_LEVEL
• VBV_BUFFER_SIZE
• DE_BLOCKING_FILTER
• B_PIC_PATTERN
• MAX_NUM_REFRAMES

I realize that I am asking a lot of you, so thank you again, so much! I'll do my best to help out with testing before this goes into a beta/pre-release build.

[jp9000]

I can implement any command, just note that any feature that isn't currently available with FFmpeg might not work if someone has to fall back to the FFmpeg version for whatever reason (if OBS is using the wrong texture format or if OBS is not running on the same GPU as the encoder). I could probably write a custom FFmpeg patch to make them work with FFmpeg but I'm not entirely sure how I feel about doing that.

Some of those commands are probably just standard FFmpeg commands that I'm missing, in which case are no problem, but I'm pretty sure some of the ones you specified are not, and are features that I'm pretty sure are not currently implemented in the FFmpeg version (the FFmpeg AMF encoder appears to be somewhat limited unfortunately).

My question to you is why specifically do you want to modify specific detailed features directly rather than just use, say, the presets?

[cs9kc]

My question to you is why specifically do you want to modify specific detailed features directly rather than just use, say, the presets?

After I made my post yesterday, I did feel bad for asking of you what I did. I'm asking you to do the work that AMD themselves should have done a long time ago with ffmpeg; but they have not yet done so.

I will retract my request for most of the features for now, all that your encoder needs to do is work and work properly. If I could only request ENCODER_PEAK_BITRATE, B_PIC_PATTERN, and MAX_NUM_REFRAMES, I would appreciate it.

I personally have made my complaints known to AMD themselves, and early on, I thought your actions were the result of those complaints, but alas, it does not appear so. I will keep pressing AMD to get involved with ffmpeg and OBS Studio.

Thank you again for all that you've done so far, @jp9000! <3

[jp9000]

Don't worry, I did not take you as being demanding at all, so no need to feel bad. There's absolutely no problem asking. I want to make sure people are happy with it. My concerns are mostly just making sure the fallback works properly more than anything.

[cs9kc]

Don't worry, I did not take you as being demanding at all, so no need to feel bad. There's absolutely no problem asking. I want to make sure people are happy with it. My concerns are mostly just making sure the fallback works properly more than anything.

Thank you for your understanding, that does mean a lot <3

The big things I would like to see are "PreAnalysis" (not the legacy Pre-Analysis that got renamed to PreEncode), and the options that I mentioned here:

I will retract my request for most of the features for now, all that your encoder needs to do is work and work properly. If I could only request ENCODER_PEAK_BITRATE, B_PIC_PATTERN, and MAX_NUM_REFRAMES, I would appreciate it.

Anything past the listed options here are just icing on the cake. Thank you again!

[flaeri]

B_PIC_PATTERN (number of b-frames) might be nice to have, as its something we expose for nvenc.
It should be accessible from ffmpeg as well (bf=x). Not all GPUs have support for this, and ffmpeg gracefully fails to 0. Hopefully AMF just handles this?

MAX_NUM_REFRAMES I dont think is accessible in ffmpeg, and they seem to just default to 1, regardless of profile or level, at least on my GPU.

[jp9000]

Well, regardless of whether they're supported by FFmpeg or not, I not only added preanalysis option, I also added the ability to specify custom AMF options in addition to FFmpeg options; however they will only work when using texture-amf, i.e. when it's not falling back to FFmpeg. So as long as you know the name of the AMF parameter, you can set it. For example, BPicturesPattern=3 will be the equivalent of directly setting the AMF_VIDEO_ENCODER_B_PIC_PATTERN to 3.

So basically you have full control over the API with this. Only problem is you can only pass in integers/bools. Not sure if any of the parameters are floating points or anything. Probably wise to be careful with it overall.

[jp9000]

When someone has a chance, please feel free to test the new commit out and let me know if you're happy with it and if it's working okay for you.

[cs9kc]

So basically you have full control over the API with this.

Oh, oh wow. Thank you!

Only problem is you can only pass in integers/bools. Not sure if any of the parameters are floating points or anything. Probably wise to be careful with it overall.

Understood. And yes, this could potentially get people into trouble if they aren't familiar with how some AMF features interact with others. Thank you for adding it, though! Perhaps a text warning with something along the lines of "use at own risk; no support" in the pop-up text would cover you from the masses if they have issues using that mode.

Thank you SO much! I will text it thoroughly over the coming days!

[AdelKS]

The texture-based encoder is Windows only since it also rely on DirectX.

Oh, I understand better. obs on Linux runs on opengl, so I suppose to support Linux, this PR would need to implement the part where it retrieves opengl surfaces instead of directx ones, and feeds them to the AMF encoder. It would be awesome if that could be done.

Thank you for your answer.

[macchky]

I can test on RX6700XT.
Can anyone upload latest windows binaly?

[theHamsta]

@macchky you can find the artifacts for each push when clicking on "details" next to the tests: https://github.com/obsproject/obs-studio/actions/runs/2500091823 you should be able to find the build for win64 in this link

[lextra2]

@jp9000
The gui for Keyframe Interval is completely broken (tested with rx 5700)
First of all, the first time you load the plugin, it will default to 250 frames. Now, if you dare to enter any value, doesn't matter which value, it will get stuck at 120 frames. (only tested with cbr. seems to be hardcoded to 2seconds?)

The only way to use a custom keyframe interval right now, is to use IDRPeriod=

Another thing is, you've enabled vbaq by default (?)
It might cause some visual issues. See GPUOpen-LibrariesAndSDKs/AMF#330

Some documentation on which settings are enabled/disabled by default would be very welcome.

[macchky]

Testing latest binaly on 6700XT, it seems to be high encoder load with b-frame for 1080p60.
On taskmanager's GPU tab (Video Codec 0) stick to 100%, and recorded movie is picture‐story show.

936p60 are almost sticking 100% but smooth video recorded.

Used options: MaxConsecutiveBPictures=2 BPicturesPattern=2 EnablePreAnalysis=true HighMotionQualityBoostEnable=true MaxNumRefFrames=3 BReferenceEnable=true QuarterPixel=true

[Toniob]

Hello. I’ve tested the build on my RX 6650 XT, with the 22.6.1 drivers, and it’s working fine at the moment. I’m using AMF only for streaming on twitch, not for the local record. But I don’t know where to find the Options I can use. Is there a documentation somewhere ? Is there options you would recommend for Twitch streaming (at 1080p50) ?

[macchky]

@Toniob You can look here
https://github.com/GPUOpen-LibrariesAndSDKs/AMF/blob/fbf12cd39fe1812ed902525a1c001307b94871b9/amf/public/include/components/VideoEncoderVCE.h#L156

and look my comment above.

[FreezyLemon]

I'm getting <Id: 1> Unable to set converter transfer characteristic, error AMF_NOT_FOUND (code 11) on initialization. The reason seems to be that in AMF version 1.4.18,

the macro AMF_VIDEO_CONVERTER_TRANSFER_CHARACTERISTIC was split into AMF_VIDEO_CONVERTER_INPUT_TRANSFER_CHARACTERISTIC and
AMF_VIDEO_CONVERTER_INPUT_TRANSFER_CHARACTERISTIC.

This branch is compiled against 1.4.14, so this causes transfer characteristics to not be set correctly when using a runtime version >=1.4.18.

Keyframe interval works correctly for me when setting via GUI. GPU is 6900XT.

[lextra2]

Keyframe interval works correctly for me when setting via GUI. GPU is 6900XT.

Really? Does it say, 180 frames in the log with 3 seconds at 60 fps? That is good to know...

[FreezyLemon]

I checked again, and yes, it definitely works. ffprobe confirms that the encoder inserts an I-frame every X seconds (X = setting from GUI). So for example, recording at 60fps with keyframe interval 7 seconds, it does one I-frame, then 419 P-frames (maybe B-frames if you enabled them) and then another I-frame.

[jp9000]

@lextra2 I fixed the keyframe interval issue with HEVC. Was using the wrong property, sorry about that.

[jp9000]

Going to merge this for now just because we really need to start further testing soon, if there are any further issues I can fix them later.

I fixed the issue with the HEVC GOP size, and I believe I fixed the issue with older devices that would use the quality preset when they couldn't quite support the quality preset; if quality won't work it should fall back to balanced, or speed if necessary.

If anyone has any further issues let me know.

[nwgat]

do this enable the new b-frames in latest amf release?
https://codecalamity.com/amd-re-introduces-the-b-frame/

[flaeri]

do this enable the new b-frames in latest amf release? https://codecalamity.com/amd-re-introduces-the-b-frame/

@nwgat yes, but you still need to have a supported GPU, ie RDNA1 (RX 5000 and higher)

[lextra2]

@jp9000
Could you clarify two things real quick:

1. Is PreAnalysis enabled by default?
2. Is VBAQ enabled by default?

[flaeri]

@lextra2
VBAQ is enabled by default.
pre-encode assisted rate control is also enabled by default.
preAnalysis is not set, so the default AMD uses would be in effect, which I believe is disabled (based on the info in the header).

[lextra2]

@flaeri
Thanks!

[Kobi-Blade]

@lextra2 VBAQ is enabled by default. pre-encode assisted rate control is also enabled by default. preAnalysis is not set, so the default AMD uses would be in effect, which I believe is disabled (based on the info in the header).

preAnalysis is too slow for streaming, so it should remain off by default, just saying.

[lextra2]

@lextra2 VBAQ is enabled by default. pre-encode assisted rate control is also enabled by default. preAnalysis is not set, so the default AMD uses would be in effect, which I believe is disabled (based on the info in the header).

preAnalysis is too slow for streaming, so it should remain off by default, just saying.

Depends on your settings. I know someone with an r9 280x (2013) gpu that uses pre analysis with h264 1280x720p 60fps 8k bitrate, quality preset. Encoder load is around 70% on that gpu. It works just fine.