SmartCut is not frame accurate

[pikim]

SmartCutting has been improved in this version, but two issues remain:

1. The middle segment of a clip ends two frames too late or the end segment of this clip starts two frames too early. This means that two frames are doubled inside the result.
2. The end segment of a clip ends about three frames too early.

There seems to be a problem with ffmpeg seeking. While the overall seeking accuracy was improved by using combined seeking, the above issues remain although the timestamps are absolutely correct.

Seeking offsets between MPV and ffmpeg (see ozmartian#263) could cause [2], but not [1].

[major-gnuisance]

There seem to be some combined seeking inaccuracy problems in ffmpeg that may go away by adding -ss 0 as an output option in addition to the -ss input option.

I'm not sure where I saw that or why it works, but take this example using some example footage recorded with OBS Studio (30fps, h264+aac)

$ ffmpeg -v 0 -ss 0.033 -i 'example.mkv'           -t 1 -c:v libx264 -c:a copy -y out_1.ts
$ ffmpeg -v 0 -ss 0.033 -i 'example.mkv' -ss 0     -t 1 -c:v libx264 -c:a copy -y out_2.ts
$ ffmpeg -v 0           -i 'example.mkv' -ss 0.033 -t 1 -c:v libx264 -c:a copy -y out_3.ts
$ diff -s out_1.ts out_2.ts 
Binary files out_1.ts and out_2.ts differ
$ diff -s out_2.ts out_3.ts
Files out_2.ts and out_3.ts are identical

Additionally:

$ ffprobe out_1.ts 2> 1
$ ffprobe out_2.ts 2> 2
$ diff 1 2
13,14c13,14
< Input #0, mpegts, from 'out_1.ts':
<   Duration: 00:00:01.03, start: 1.433667, bitrate: 2754 kb/s
---
> Input #0, mpegts, from 'out_2.ts':
>   Duration: 00:00:01.00, start: 1.466667, bitrate: 2836 kb/s

So out_1.ts has one more frame's worth of "duration" and one less frame's worth of "start", whatever that means.

However, if I omit or transcode the audio, this doesn't seem to happen:

$ ffmpeg -v 0 -ss 0.033 -i 'example.mkv'           -t 1 -c:v libx264 -y out_1.ts
$ ffmpeg -v 0 -ss 0.033 -i 'example.mkv' -ss 0     -t 1 -c:v libx264 -y out_2.ts
$ ffmpeg -v 0           -i 'example.mkv' -ss 0.033 -t 1 -c:v libx264 -y out_3.ts
$ diff -s out_1.ts out_2.ts 
Files out_1.ts and out_2.ts are identical
$ diff -s out_2.ts out_3.ts
Files out_2.ts and out_3.ts are identical

Maybe the accuracy claim of input -ss requires that all streams are being transcoded, and the audio being stream copied is confusing things? (I suppose AAC has a similar problem to h264 GOPs and can only be cleanly split at specific sites.)

Still, why does adding an output -ss 0 seem to make a difference?

Note that the current release (6.0.0) is only using output -ss in beginning/end segments, and I think it's being frame accurate, albeit infuriatingly slow with larger files due to slow seeking.

btw I found all this while trying to quickly hack support of input -ss into VidCutter, which I just discovered a few hours ago.

Note: results above with ffmpeg version 4.1.6-1~deb10u1 (Debian stable)