obs-outputs: Add dynamic variable bitrate

[pkviet]

Suggestions link : here

Updated (5/5/2018)

Partly based on R1CH code from OBS Classic with some changes from Woodbyte fork.
I'm removing the experimental tag since it's working quite well.
The algo has seen a series of improvements, notably support for all h264 encoders supported by obs
(x264, enc-amf, nvenc, mac-vth264, quicksync, etc).

warning

• For enc-amf, a flag signalling that dynamic bitrate must be added; I put this as a placeholder so that I don't forget to submit a PR to enc-amf submodule repo. (see below update of 02/22)
• for (old) nvenc, ffmpeg master git head is required since dynamic bitrate was added to nvenc ffmpeg only recently (05/05/18 ) through a patch I submitted.
• new jim-nvenc supported. Works much better than previous nvenc. I routinely get zero frame drops.

What this does:

• for rtmp streams, decrease the video bitrate when available bandwidth is going down until all packets go through;
• increase the video bitrate once bandwidth conditions get better.

Some screenshots
(1) enable dynamic bitrate in Simple output

(2) Advanced settings for dynamic bitrate

(3) Bitrate decrease from 2500 kbs due to congestion (downward red arrow)

(4) Bitrate stationary during congestion (not increasing yet because congestion hasn't cleared, arrow is now orange and not red as in this screenshot, see discussion with Dodgepong)

(5) Bitrate increasing due to congestion clearing (upward green arrow (red in this screenshot))

[RytoEX]

Hi! I've reviewed this PR.

Normally, I would wait to do a style review on a PR that's actively seeking feedback or reviews on functional stuff, but I've found that if I put it off, I'll just forget.

There's a few other places that I haven't commented on yet for long lines, partly because they might get refactored, and partly because I'm not certain how best to line break them.

In addition to my Consider putting blank lines before and after if/else blocks to make them stand out better from the surrounding code.

Lastly, please change the commit subject's prefix to "obs-outputs" and make sure the commit subject starts with a capitalized verb, perhaps like this:

obs-outputs: Add dynamic variable bitrate

[pkviet]

thanks again !

[pkviet]

On Hold

Following a suggestion of Xaymar, I'll add enc-amf support. For now this plugin works only for x264.
I'll try to add also for nvenc and libmfx (qsv)

Edit:
support for qsv and enc-amf added.
Nvenc NOT supported (dynamic bitrate is supported by nvidia api but this does not seem implemented in ffmpeg).
In testing phase for enc-amf.

[pkviet]

PR Updated

Support for enc-amf and QSV added.
FFmpeg-nvenc is unfortunately incompatible with dynamic bitrate; the feature was not implemented.
In terms of bitrate switching, the most efficient encoder is QSV ; enc-amf and x264 are on par, one better than the other in some conditions.
The switching has been tested under hard-wall conditions: upload speed bounded (through a router),
and on macOS by simulating various internet connexions using Network Link Conditioner (free XCode tool).

[pkviet]

Update and to do

• Added flags to h264 encoders ( through info.caps ) to signal dynamic bitrate support (cf @reboot ).
• Added log when dynamic bitrate is enabled (cf @dodgepong )
• Changed text displayed in UI for option (cf @notr1ch )
• Added an arrow in status bar to indicate whether bitrate switching is active and which way it is going (increasing bitrate, decreasing it or leaving it) (cf @dodgepong on discord )
• TO DO: enable finer user control on bitrate decrease or increase rate in UI (cf @jp9000 on discord who thought the decrease rate is quite steep; the right rate much depends on local network conditions so this would allow the user to adapt it a bit).
  I'll push the additional commits when the last item is done.
  I'm separating the features in different commits for easier cherry-picking.

edit: last task done (02/22/18)

[pkviet]

Update (02/22/2018)

Applied the suggestions in the various reviews.

NOTE (reminder):
in plugins/enc-amf/Source/enc-h264.cpp#LN101
add: encoder_info->caps = OBS_ENCODER_VIDEO_DYN;
will submit a PR in submodule later if this PR is merged.

[SuslikV]

1. I don't see the way to modify the icons for my custom theme (.qss). Is it even possible?

2. I have other thoughts for description text (the difference info).

Basic.Settings.Output.DynamicBitrate="Auto manage bitrate decrease in case of network congestion"

• (otherwise, I think that this option do miracle and auto-adjust my bitrate as magic Auto-Configuration Wizard. The encoder specific {NVENC} shouldn't be mentioned - you need to disable/hide that control - that's all.)

Basic.Settings.Output.DynamicBitrate.Tooltip="If enabled and network congestion is detected, video bitrate\nwill be decreased by few % every second.\n\nOnce the congestion has cleared up, bitrate will increase every 5 sec,\nuntil target bitrate reached."

• (not clear, should I leave this option unchecked or enable it? Exact numbers (%%) not needed because they are subject to change and not adjustable, or you should give detail description at what % will decrease/increase each encoder. I mean - "~about" is not an option here.)

Basic.Settings.Output.Adv.DynamicBitrate="Auto manage bitrate decrease in case of network congestion"

• (same as Basic.Settings.Output.DynamicBitrate)

Basic.Settings.Output.Adv.DynamicBitrate.Tooltip="If enabled and network congestion is detected, video bitrate\nwill be decreased by few % every second.\n\nOnce the congestion has cleared up, bitrate will increase every 5 sec,\nuntil target bitrate reached."

• (same as for Basic.Settings.Output.DynamicBitrate.Tooltip)

Basic.Settings.Output.Adv.Dyn.Down="Decrease bitrate, step (in %)"
Basic.Settings.Output.Adv.Dyn.Up="Increase bitrate, step (in %)"

• (of course, if rate is mentioned this should change only rate - because previous option says about bitrate, "stepping" - is option too, otherwise just leave it unchanged)

Basic.Settings.Output.Adv.DynamicBitrate.Down.Tooltip="When congestion is detected, the video bitrate will be decreased\nevery second by the specified amount until congestion stops."

• (I don't get what is "X %" here. If this variable then syntax is wrong. If not - leave only brief info here. Defaults may differ per encoder, if not - add \n\nDefault value is 15%. at the end)

Basic.Settings.Output.Adv.DynamicBitrate.Up.Tooltip="When congestion clears up, the video bitrate will be increased every\n5 seconds by the specified amount until target bitrate reached."

• (I don't get what is "X %" here. If this is variable then syntax is wrong. If not - leave only brief info here. Defaults may differ per encoder, if not - add \n\nDefault value is 10%. at the end)

3. I want to know (from interface) how much it can decrease the bitrate (read - quality of the stream) when enabled, until frame dropping will be forced. Maybe you should add this to tool tip of the option as \n\nMinimum possible bitrate is half of the target bitrate,\nif congestion continues - frame drop will be forced.

[pkviet]

@SuslikV

1. check in UI/forms/images for the icons names. Then look at UI/data/themes/Rachni to see how you can override the icons.
2. Disabling the option for nvenc requires some UI work which I hesitate to implement at the moment because it will have to be reverted if ffmpeg-nvenc gets dynamic bitrate support (I'm thinking of submitting a patch to ffmepg eventually). But I'll look into it.
   For the tooltips, texts, let me think about it and also in light of further reviews.
3. The dynamic bitrate mechanism is independent from the frame drop mechanism. It sets in much sooner than frame dropping so it can prevent frame dropping altogether if aggressive enough, though in general it won't prevent frame drops but will mitigate them.
   So it's not possible to quantify when frame dropping will start as a function of bitrate decrease.

[SuslikV]

1. I don't get it. The Rachni alternates only general icons for widgets, not the one that is from the storage (obs.qrc). And few, that has custom string properties (like "themeID"). Can you explain more, how to access your images by Qt stylesheet?
2. As you wish.
3. Queue shortens faster? The bitrate has minimum? How can you explain to the end user why dynamic bitrate is needed then? If this feature not needed - no sense to implement it. For example, I know, if I have a lot of drops - I need to lower bitrate - it always helps.

[pkviet]

1. look at the list of files in that PR: there are 3 png, dyn.png, dynup.png and dyndown.png ;
   once merged they will be located in UI/forms/images ; if you check obs.qrc of this PR you will see there are three new entries.
2. if the user has say 1 Mbs of upload bandwidth but selects more, the option will lower automatically the bitrate for him/her. If the upload bandwidth fluctuates , it will do also. And when bandwidth gets larger, the option will increase back the bitrate.
   If the bitrate is contained, it will of course decrease the number of frame drops. The option controls frame drops indirectly in this way: if bitrate decreases, buffer will not be as filled and therefore frame drops will be less frequent.
   When I say the option can only mitigate drops and not prevent them completely is because when the algo tries to increase the bitrate, it will test against congestion parameter (which is a measure of how filled is the buffer): if there is no congestion, the algo knows it can still increase the bitrate until congestion appears. Then as to frame drops occuring during that increase phase, it all depends on a combination of how fast network bandwidth fluctuates and how fast the algo adjusts bitrate. When decreasing bitrate, it is the same; suppose bandwidth is going down faster than the bitrate adjustment, the buffer will get filled and therefore there will be drops. But less than if the option had not been activated.
   What must be realized is that frame drops / dynamic bitrate are two different mechanisms to deal with fluctuating bandwidth. The first prioritizes the quality of the picture, with less frames; the second prioritizes the frames continuity to the expense of image quality.

[SuslikV]

Oh, about the theme icons, I mean - custom (user) themes, for example: https://obsproject.com/forum/resources/categories/themes.10/
I thought, that it would be nice to have styled elements when you make changes to UI and implementing this new code in obs. And current implementation as hardcoded strings in the code - they are exist but you cannot adjust them without bit-hack.

The description is nice. Unfortunately it is huge. And thus tool tip with all this info is a bad idea. Also, user may don't know about the encoder's buffer, it fullness etc. Let's imagine that All what I (user) see is: auto option; current bitrate value; up/down arrows; frame drop or quality drops (from stream watchers feedback). And all this, is connected to user settings of the mentioned option (no matter how it works).
Now I thinking of naming the option: "Dynamic bitrate reduction" this is short and all other info can be moved into the tool tip.

[pkviet]

1. Themes can override all icons, so these icons in my PR can also be overriden . But maybe I misunderstood you, sorry then. Might be simpler to discuss this on discord on dev channel.
2. You're right , the end-user does not need too much info. Notice the controls for the algo only appear in Output > Advanced mode.
   I get your point about tooltips and the rest. I'm waiting for Jim's review to synthesize everything now. Thanks.

[SuslikV]

In general I don't know how to select exact object. QLabel is selectable but all 3 states (up/down/static) is QPixmap picture that replaces each other. Thus I can give only one style to QLabel itself but not designate it states. After picture updates, the style needs to be updated in the code too (at least for this element), the simplest way is to set style with empty comment string. Also, to be able to address each state I need unique name for it (the simplest way is to add string to properties, for example "themeID"="up_bitrate") this name should change as soon as new state switched, but before style update. Thus it will be possible to use .qss:

QLabel[themeID="up_bitrate"] {
    my custom style...
}

Also, it is desirable to have unique and predefined "name" property for the widget (QLabel). Because potentially you can share same picture to other widgets and adjust only size of it for each widget type individually.

The chance to be implemented is small because I'm talking about the changes that were not accepted. It is different, but the root is the same. #856

[pkviet]

Update 03/14/18: fixed some UI bugs (never use Qt Creator on obs ui files !)

[dodgepong]

Upon discussion in the discord, I actually think this feature should be enabled by default. For the average user, I think they would prefer to degrade quality and not drop frames rather than drop frames with static quality.

[pkviet]

Update:
Fixed mem leaks.

[twilson90]

I found using the default NVENC encoder rather than the one marked (new) didn't work as desired with dynamic variable bitrate.
It would constantly flit between the lowered bitrate and the normal high bitrate specified in the encoder settings (CBR), causing frame droppage during congestion.
However, selecting the (new) NVENC encoder fixed this issue.
I'm curious to know if anyone else has experienced this.

[twilson90]

This looks great but how does one download this for use?

I just compiled it for 64 bit windows: https://drive.google.com/file/d/17tO5KRQaSUFxcmlUtYVdagbcXazrc3VU/view

[pkviet]

I found using the default NVENC encoder rather than the one marked (new) didn't work as desired with dynamic variable bitrate.
It would constantly flit between the lowered bitrate and the normal high bitrate specified in the encoder settings (CBR), causing frame droppage during congestion.
However, selecting the (new) NVENC encoder fixed this issue.
I'm curious to know if anyone else has experienced this.

Firstly, it's true that the new nvenc is better than the other encoders as to dynamic bitrate.
For the other encoders (x264, old nvenc, amd amf), you need to tighten the params:

• set for instance 100 ms for Decrease polling time instead of the 300 default
• increase the 'Bitrate decrease rate' (ex: 25% instead of 15 %).
  In that way with CBR, old nvenc I completely avoid dropped frames.
• i'm gonna set the minimum threshold for bitrate drop to 1% (instead of 5% currently)

[pkviet]

update

• changed the default settings to decrease bitrate more aggressively ; will play better with old nvenc and x264
  (see discussion with @hedgehog90 )

[Ciommi]

If some pro can compile every new Version of Obs with this fantastic feature would be fantastic for nabs people like me 😋 thx anyway and amazing work

[pkviet]

update 03/30/19

• major styling fixes ==> strict limit to 80 chars/line, shortening of var/function names, fix usage of CamelCase vs snake_case,
• tried to improve readibility by splitting large functions (less 'squishy' code hopefully)
• added defaults to rtmp output instead of setting them in window-basic-main for Simple Output.
• removed long comments (wall of text).
• changed defaults (the 30 sec for increasing the bitrate is too long; this would pin the bitrate to lower values longer).
• mean congestion is computed now as a moving average , so I dropped all congestion arrays, which simplifies the algo.

For the sake of comparison I've left in the code an implementation of R1CH algo (adjust_bitrate-r1ch) written by Jim; it will have to be removed eventually.

Here's a comparison with R1CH original algo and the PR algo:

• in general, R1CH algo has more dropped frames (with jim-nvenc: x1.5, ffmpeg-nvenc: x2, x264: x2-x3) for a similar average bitrate.
• R1CH algo has a floor (min bitrate) set quite low at 100 kbs; for such a value, it has in general an average bitrate which is higher than the PR algo. But if one sets the floor for the PR algo to more realistic values, the average bitrate is similar.

For details of the comparison between the algos with screen captures, check: https://docdro.id/Z5lwYX8

[Krutonium]

This hasn't been touched in a month, could this be re-reviewed please? I have been using this build of OBS for a while now and I'd love to be able to switch back to mainline.

[dodgepong]

It hasn't been touched because work has been more on getting 23.2 out the door. That said, Jim has been doing quite a bit of testing and tweaking with the algorithm internally lately. No need to be impatient.

[Krutonium]

@pkviet I just found a crash that appears to be specific to this version of OBS - Specifically, when using QSV on an Intel HD 3000 (i7 2630QM) Laptop. This crash does not occur on the current versions of OBS.
Crash.txt

It also crashes when using FTL instead of RTMP.

[jp9000]

For clarification I'm not implementing it myself, I just modified the algorithm. I'll be contributing to this PR and adding some updates. I'll also update it for clang-format.

[pkviet]

feature coded by Jim with improvements has been merged so closing this.