[stable23] Fix flickering with background blur when changing quality #6586
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![backportbot-nextcloud[bot]](https://avatars.githubusercontent.com/in/21012?s=60&v=4){kind=small}
Whenever a segmentation is calculated it is used to draw the input stream to the output canvas with the background blurred. The segmentation calculation was triggered at regular intervals, but it was not taken into account whether a previous segmentation was still being calculated. Now a new segmentation is calculated only if the previous one finished already. This prevents running outdated segmentations and thus should reduce the load on low end devices in which calculating the segmentation for a frame takes longer than the time elapsed between frames. Besides that, if the input stream was changed the previous segmentation could be processed when the new one had not finished loading yet, which could led to visual glitches. This is also implicitly fixed by this change, as the previous segmentation will be discarded if it does not match the expected frame id. Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
The segmentation was always run at 30 FPS. However, if the input stream has a lower FPS calculating the segmentation again is a waste of resources, as it might have not changed since the previous input frame. Moreover, as the output stream also runs at the same FPS as the input the canvas could be drawn without any effect in the output. As the input stream, the calculation of the segmentation and the drawing of a new output frame are not synchronized in some cases this could introduce some lag between the drawn segmentation and the input video. However, due to the lack of synchronization that could happen already (and the fixed 30 FPS only overcomed that on lower FPS inputs by sheer luck and brute force), and the reduced load is worth that minor annoyance. This could be solved by synchronizing the input stream and the segmentation mask, but that comes with its own challenges. Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
When the constraints of the input stream were changed the background blur was stopped and started again to apply the new width, height and frame rate to the output. However, changing a stream (any stream, it is unrelated to the background blur) causes a flickering (one or more black frames in between), so every time that the sent video quality was adjusted the local video flickered. Now, instead of reseting the background blur, the internal elements are updated and adjusted to the new constraints of the input, but the same output stream is kept. This avoids the stream change and thus the flickering. Note, however, that the video will temporary freeze instead while the input stream is being re-rendered, although this should be less annoying than the flickering. Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
backportbot-nextcloud
bot
added
bug
feature: call 📹
backportbot-nextcloud
bot
requested review from
danxuliu,
ipasanec,
marcoambrosini and
nickvergessen
nickvergessen
approved these changes
Nov 22, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
backport of #6569