Recording: Don't use stateful filters in ffmpeg video processing #16047
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Because the input videos for BigBlueButton recording processing switch resolution and aspect ratio, the filter chain gets re-initialized, and any state in the filters is lost. This causes problems with the following filters:
color
: Timestamps restart from 0, rather than continuing at the point where they left off.fps=start_time=12.345
: After reset, the fps filter thinks it's at the start of the file again, so the next frame it sees gets duplicated output for timestamps from thestart_time
until it catches back up.setpts=PTS-STARTPTS
: The 'STARTPTS' is re-read as the first pts the filter sees after reinitialization, so timestamp of the next frame is reset to 0. (In practise, this didn't cause any problems because the duplicate frames created by the fps filter had the original start time.)The end result of all of these issues is that a lot of duplicate frames were created with invalid timestamps, which eventually get discarded by ffmpeg. But a lot of time is wasted, causing recordings to sometimes take hours to process when they should be ready in minutes.
The final fix is to split out some of the video filtering to a separate ffmpeg process, which is then piped into the main compositing process.
The separate ffmpeg process is responsible for:
And the main compositing ffmpeg process can then do the stateful part: