Va-API code, what will it take to get this merged?

[rigred]

Based on patch found here, with minor tweaks.

https://raw.githubusercontent.com/GloriousEggroll/arch-obs-studio-latest-ffmpeg-vaapi/master/ffmpeg-vaapi.patch

Ultimately based on the fork here:

https://github.com/w23/obs-studio/tree/ffmpeg-vaapi

[Fenrirthviti]

Well, reading the contribution guidelines is probably a good place to start, since this is kind of a bad PR.

There's also another PR already open about this, which can be found here: #941

I would recommend you close this one and move any discussion/updates to that PR instead.

[jp9000]

No no, Fenrir -- this is honestly probably the best way to do it, through FFmpeg. The other branch implementing it ourselves is likely always going to be more of a waste when FFmpeg already has an implementation. If I can get some tests rolling on this I think I'll be likely to merge this at some point.

[rigred]

Thanks.
I am simply boldly going for it to find out what's going on with this feature.

It obviously it needs adaptation to flesh it out and work out the edge cases and some likely bugs.

EDIT: Ok not completely stable. Noticed some odd stuttering when recording and the framerate being stuck at 1000 in the recording file.

So this is just a start.

[rigred]

Refer to the linked reference above by GloriousEggroll.
This requires very new mesa 17.3 with amdgpu and va-api extensions.

So everything is WIP.

[GloriousEggroll]

this is not ready for upstream yet. it requires at least mesa 17.3, and the vaapi amdpu featureset is not near complete yet, some features are not fully functional. Also I do not have any stuttering or anything when i record using this. I've used it for both recording and streaming. Some tips:
When streaming you have to use baseline profile or twitch wont recognize it. main and high also work for recording. I do not have the framerate issue you had either. Make sure your keyframe interval is set to 2 and bframes and quality are both set to 0 for amd gpus. Currently running mesa 17.3.2

[jp9000]

Wait you grabbed this from someone else's repository? @GloriousEggroll was this yours originally? Sorry, I didn't even realize. Could you make a PR for this when you're ready with it? I'm going to close this until the actual author of the FFmpeg VAAPI encoder implementation is ready to submit it. But I will be very likely to merge an FFmpeg implementation of this if someone manages to get it working without any major issues.

[rigred]

Yes @jp9000 I mentioned in the first message that this is a repo based on an adapted patch generated from: https://github.com/w23/obs-studio/tree/ffmpeg-vaapi

via GloriousEggrolls' patch.

I claim no credit for this. It's simply an attempt to get the ball rolling on getting any sort of hardware accelerated enconding on linux for non nvidia users.

[GloriousEggroll]

w23 created the base patch just to get h264 working on the default system gpu, so originally it was his work. I updated it to get it working with mesa, allow selecting of gpus, allow multiple codecs (hevc, h264, mpeg2, vp8, vp9), profiles, keyframes, bframes, levels, quality,render modes, etc, all the necessary features.

some of these features however, while implemented, are incomplete in mesa, in particular on the amdgpu side.

Ideally you could remove some of the features: force h264 codec, baseline profile, bitrate render mode, and remove the bframe and quality settings, and this would work for most users

[jp9000]

Alrighty! Well once you feel it's ready, feel free to make a pull request for it. Thank you (to both of you) for doing that by the way.

[rigred]

Resolved the framerate oddity by clearing OBS settings back to default.
Unfortunately not sure what caused that.

Still have abysmal framedrop at higher resolutions and framerates.
Bitrate is set sufficiently above the needed rate. But doesn't seem to have any impact.

My system:
Dual Rx580 + Ryzen 1700X.
Arch Linux 4.14.13 + latest amdgpu/mesa 17.3.2
Doesn't matter on which GPU I encode, the framedrop is there.
Also happens on my other Rx 550 machine.

Settings:
1080p60fps
h264 baseline
bitrate 10'000/40'000/80'000
keyframe interval 2.

Will test further.

UPDATE: No framedrop at:
720p 60fps.
1080p 30fps

EDIT:
Understood! jp9000

[steffenWi]

@rigred regarding framedrop: I haven't tested it with the 21st version but on my RX 570 with amdgpu/radeonsi I can go up to 1080p@48fps with a bitrate of ~15000 without getting framedrops.

If I go higher, either with the FPS or the bitrate I get framedrops as well. Interestingly this has no impact on the game (no reduction in performance) and no impact on the CPU either (it's not running at full capacity).

The only 'hint' I get is that in the status bar on the bottom of the OBS window it will, after ~10 seconds of recording, say something like that the encoder is overloaded and to reduce settings to avoid framedrops.

I'm on Arch Linux as well just for the record.

[fosspill]

Vega 56. Kernel 4.15:
1080p60fps
h264 baseline
bitrate 15000
keyframe interval 2

1 frame dropped.

edit:
Same settings, but 25k bitrate. ~7 dropped frames:
info: Output 'adv_file_output': Total frames output: 2168
info: Output 'adv_file_output': Total drawn frames: 2175

Dropped under 60 fps for a very short time while recording. I'm guessing this happens when OBS isn't getting enough frames from whatever it is recording?

[dR3b]

This works really well! Thanks!

info: OBS 21.0.3_17_g7337b7ff (linux)
info: ---------------------------------
info: ---------------------------------
info: audio settings reset:
	samples per sec: 44100
	speakers:        2
info: ---------------------------------
info: Initializing OpenGL...
info: Loading up OpenGL on adapter Intel Open Source Technology Center Mesa DRI Intel(R) Iris Plus Graphics 650 (Kaby Lake GT3e) 
info: OpenGL loaded successfully, version 4.5 (Core Profile) Mesa 17.3.6, shading language 4.50
info: ---------------------------------
info: video settings reset:
	base resolution:   3440x1440
	output resolution: 3440x1440
	downscale filter:  Lanczos
	fps:               30/1
	format:            NV12
	YUV mode:          601/Partial
info: Audio monitoring device:
	name: Default
	id: default
info: ---------------------------------

...

info: FFMPEG VAAPI supported
info: VLC found, VLC video source enabled
info: ---------------------------------
info:   Loaded Modules:
info:     vlc-video.so
info:     text-freetype2.so
info:     rtmp-services.so
info:     obs-x264.so
info:     obs-transitions.so
info:     obs-outputs.so
info:     obs-libfdk.so
info:     obs-filters.so
info:     obs-ffmpeg.so
info:     linux-v4l2.so
info:     linux-pulseaudio.so
info:     linux-jack.so
info:     linux-decklink.so
info:     linux-capture.so
info:     linux-alsa.so
info:     image-source.so
info:     frontend-tools.so
info: ---------------------------------
info: ==== Startup complete ===============================================
error: Service '' not found
info: All scene data cleared
info: ------------------------------------------------
info: pulse-input: Server name: 'pulseaudio 11.1'
info: pulse-input: Audio format: s16le, 44100 Hz, 2 channels
info: pulse-input: Started recording from 'alsa_output.pci-0000_00_1f.3.analog-stereo.monitor'
info: xshm-input: Geometry 3440x1440 @ 0,0
info: Switched to scene 'Szene'
info: ------------------------------------------------
info: Loaded scenes:
info: - scene 'Szene':
info:     - source: 'scene1 (XSHM)' (xshm_input)
info: ------------------------------------------------
qt5ct: D-Bus system tray: no
qt5ct: custom style sheet is disabled
info: adding 46 milliseconds of audio buffering, total audio buffering is now 46 milliseconds
QXcbConnection: XCB error: 8 (BadMatch), sequence: 7123, resource id: 62914623, major code: 130 (Unknown), minor code: 3
QXcbConnection: XCB error: 8 (BadMatch), sequence: 7153, resource id: 62914623, major code: 130 (Unknown), minor code: 3
QXcbConnection: XCB error: 8 (BadMatch), sequence: 7156, resource id: 62914623, major code: 130 (Unknown), minor code: 3
QXcbConnection: XCB error: 8 (BadMatch), sequence: 7176, resource id: 62914623, major code: 130 (Unknown), minor code: 3
info: ---------------------------------
info: [FFMPEG VAAPI encoder: 'recording_h264'] settings:
	qp:           20
	quality:      0
	profile:      100
	level:        42
	bitrate:      8192
	keyint:       120
	width:        3440
	height:       1440
	b-frames:     0
	loop-filter-level:     16
	loop-filter-sharpness:     4

info: libfdk_aac encoder created
info: libfdk_aac bitrate: 160, channels: 2
info: ==== Recording Start ===============================================
info: [ffmpeg muxer: 'adv_file_output'] Writing file '/home/xxx/2018-03-07 07-45-47.mkv'...
[matroska @ 0x55d9c7f942e0] Using AVStream.codec to pass codec parameters to muxers is deprecated, use AVStream.codecpar instead.
[matroska @ 0x55d9c7f942e0] Using AVStream.codec to pass codec parameters to muxers is deprecated, use AVStream.codecpar instead.
QXcbConnection: XCB error: 8 (BadMatch), sequence: 13671, resource id: 62914623, major code: 130 (Unknown), minor code: 3
QXcbConnection: XCB error: 8 (BadMatch), sequence: 13706, resource id: 62914623, major code: 130 (Unknown), minor code: 3
QXcbConnection: XCB error: 8 (BadMatch), sequence: 13709, resource id: 62914623, major code: 130 (Unknown), minor code: 3
info: [ffmpeg muxer: 'adv_file_output'] Output of file '/home/xxx/2018-03-07 07-45-47.mkv' stopped
info: Output 'adv_file_output': stopping
info: Output 'adv_file_output': Total frames output: 524
info: Output 'adv_file_output': Total drawn frames: 529
info: ==== Recording Stop ================================================
info: libfdk_aac encoder destroyed

[Entropy512]

I assume the Mesa requirements are AMD-specific?

I'll provide the log files tonight, but I had very good results with GloriousEggroll's original patch (I have not tried rigred's changes) on a machine with Intel Kaby Lake and Ubuntu 17.10 - no "special" driver or OpenGL/Mesa upgrades.