Seeking 5 seconds keybinds do not actually seek 5 seconds

[MarioMastr]

This might honestly be the weirdest issue I've ever come across, but if you try to use the right and/or left arrow keys to seek 5 seconds, they don't actually seek 5 seconds. Sometimes they seek 5, but sometimes they seek like 7 or 8, and I've even had up to 12 once.

System and IINA version:
MacBook Air M1

• macOS 14.2.1
• IINA 1.3.4 (custom built)

Expected behavior:
It should go back exactly 5 seconds.

Actual behavior:

Crash report:

mpv log:

Steps to reproduce:
Try and find a video and press the right or left arrow keys to go forward or back by 5 secs. It won't go back by exactly 5 secs.

• MPV does not have this problem.

How often does this happen?
A lot of the time.

[low-batt]

This default behavior is expected and very confusing to users. It can be changed. First some background…

By default the keys ← and → are bound to seek -5 and seek 5. These are mpv commands. They look like they seek by 5 seconds. But that is not what they do.

From the seek command section of the mpv manual:

seek <target> [<flags>]

Change the playback position. By default, seeks by a relative amount of seconds.

The second argument consists of flags controlling the seek mode:

• relative (default)
  Seek relative to current position (a negative value seeks backwards).
• absolute
  Seek to a given time (a negative value starts from the end of the file).
• absolute-percent
  Seek to a given percent position.
• relative-percent
  Seek relative to current position in percent.
• keyframes
  Always restart playback at keyframe boundaries (fast).
• exact
  Always do exact/hr/precise seeks (slow).

Multiple flags can be combined, e.g.: absolute+keyframes.

By default, keyframes is used for relative, relative-percent, and absolute-percent seeks, while exact is used for absolute seeks.

This means seek 5 is a relative forward seek, which is what we want. And that means this is a keyframes seek, which is fast, but isn't what you want if you really want to seek forward exactly 5 seconds. To do that you must use the exact keyword.

What this has to do with is the way videos are usually encoded. The common formats put a priority on reducing the size of the video file for normal forward playback. The video compression techniques used make it harder to seek to a random point in the video.

One of the compression techniques is to use a keyframe which contains a full picture of the video frame and then follow it by delta frames that only describe the differences from the keyframe. The IBM tutorial Keyframes, InterFrame & Video Compression is a good introduction to video compression.

The quick way to seek is to find a keyframe close to the requested position and display that. That is what mpv is doing. There is no industry standard keyframe interval. Whoever encoded the video can choose the interval to use. Shorter intervals increase the size of the video. We'd have to use ffprobe on the video and check to see what interval was used.

To seek to an exact spot in the video requires the player to find a keyframe and then decode frames until reaching the exact spot in the video. How long that takes depends upon the interval between keyframes and where the requested position happens to land.

Unfortunately changing the key binding to use an exact seek at this time will not work due to the open IINA issue #3710. The fix for that issue was just merged and will be included in the next release of IINA. Once that is fixed you can bind a key to seek 5 exact.

Making the situation worse, the recently upgraded versions of the mpv and FFmpeg libraries contain regressions that cause seeking by keyframes to sometimes malfunction. Last I checked those problems have not been fixed.

There is another way to enable exact seeks. From the mpv manual entry for hr-seek:

--hr-seek=<no|absolute|yes|default>
Select when to use precise seeks that are not limited to keyframes. Such seeks require decoding video from the previous keyframe up to the target position and so can take some time depending on decoding performance. For some video formats, precise seeks are disabled. This option selects the default choice to use for seeks; it is possible to explicitly override that default in the definition of key bindings and in input commands.

• no: Never use precise seeks.
• absolute: Use precise seeks if the seek is to an absolute position in the file, such as a chapter seek, but not for relative seeks like the default behavior of arrow keys.
• default: Like absolute, but enable hr-seeks in audio-only cases. The exact behavior is implementation specific and may change with new releases (default).
• yes: Use precise seeks whenever possible.
• always: Same as yes (for compatibility).

To enable exact seeks using hr-seek follow these instructions:

• Start IINA
• Click on Settings… under the IINA menu
• The settings panel appears
• On the left side of the panel click on Advanced
• Slide the Enable advanced settings toggle button to be on (blue)
• In the Additional mpv options section click on +
• A new entry appears in the table
• Double click on name and replace it with hr-seek
• Double click on value and replace it with yes
• Restart IINA to activate the new settings

With today's more powerful Macs exact seeking is pretty fast. Where you might see a slowdown is if you like to seek by holding down a key.

[MarioMastr]

Thank you very much for explaining this in a way that I was easily able to understand. Seeing as this is intended, there's no point in keeping this open.

[saagarjha]

Reopening because I think we should change this.

[low-batt]

Are you thinking an IINA setting that is tied to hr-seek? Maybe something like Use precise seeks by default on the Video/Audio tab of settings? With these possible values:

• For absolute seeks
• For audio-only
• Never
• Whenever possible

With Whenever possible being the default (overriding mpv default).

Currently seeking behavior is exposed on the Control tab in the mouse settings. The setting Seek type can be set to:

• Keyframe seek
• Exact seek
• Auto

Where Auto is an IINA invention. For the first seek exact is used. If the time taken for that seek is not less than 0.05 then IINA decides exact seeking is too slow and will fall back to seeking to a keyframe.

I believe this setting is only applied when using the mouse scroll wheel to seek.

[saagarjha]

I think seeking by 5 seconds should do exactly that, rather than something tied to keyframes. Users should not have to understand how this works under the hood.

[low-batt]

Based on past issues we also have users that really want fast seeking. We could change the ← and → key bindings to seek -5 exact and seek 5 exact. However that is harder for a user to reconfigure back to fast seeking. If we add a setting tied to hr-seek and default that to using exact seeks then that would change the behavior of all of the existing key bindings and be easier for the users who find that too slow to switch back to the old inexact but fast seeking.

[svobs]

Based on past issues we also have users that really want fast seeking. We could change the ← and → key bindings to seek -5 exact and seek 5 exact. However that is harder for a user to reconfigure back to fast seeking. If we add a setting tied to hr-seek and default that to using exact seeks then that would change the behavior of all of the existing key bindings and be easier for the users who find that too slow to switch back to the old inexact but fast seeking.

Absolutely agree. Many users seem to expect exact seeks by default and don't want to hear about the complexities of the keyframe problem or the correct mpv syntax.

Would also be amazing if SeekOption.auto could be applied to all seeks somehow. Either in its current form based on speed, or something that could be configured to guarantee some minimum precision (e.g., use exact seeks only if the video has a keyframe interval > 5s).

[krackers]

use exact seeks only if the video has a keyframe interval > 5s

My understanding is that keyframe interval is not guaranteed to be fixed though, if the encoder chooses it can vary (e.g. for not-changing sections you will have fewer keyframes in a given duration of time). Moreover unless demux cache is enabled I think you won't even know beforehand if the region you are seeking into contains a close-enough keyframe or not.

[low-batt]

The SeekOption.auto code is found in the seek method in PlayerCore and looks like this:

    case .auto:
      // for each file , try use exact and record interval first
      if !triedUsingExactSeekForCurrentFile {
        mpv.recordedSeekTimeListener = { [unowned self] interval in
          // if seek time < 0.05, then can use exact
          self.useExactSeekForCurrentFile = interval < 0.05
        }
        mpv.needRecordSeekTime = true
        triedUsingExactSeekForCurrentFile = true
      }

If I am understanding this code correctly it is a heuristic that bases its decision on the time taken by a single exact seek. The trouble with this is that the time taken for an exact seek depends upon where the position to seek to lands in relation to keyframes. If the position happens to land right on a keyframe then it is just as fast as a keyframe seek. This could happen with a video that has a large keyframe interval.

And of course this heuristic may come up with a different answer the next time the video is played.

[svobs]

My understanding is that keyframe interval is not guaranteed to be fixed though, if the encoder chooses it can vary (e.g. for not-changing sections you will have fewer keyframes in a given duration of time). Moreover unless demux cache is enabled I think you won't even know beforehand if the region you are seeking into contains a close-enough keyframe or not.

I'm no video expert by any means, and admittedly I was just throwing out half-baked ideas to stimulate discussion. Any strategy might have to differ completely for some encodings, and might have to resort to fuzzy logic and/or collect data as it goes along in order to make a "best guess" about the which choice to make.

But I made that suggestion under the impression that (if not 100% deterministically) certain assumptions could be made about whether a video has a roughly constant keyframe interval based on its codec type or info in its headers. I have seen that most videos I've played seem to have a pretty constant interval, but I realize I might have leaped a bit there?

Also, I wasn't suggesting trying to guess exactly where each keyframe was in relation to each seek; just the interval itself. The goal would be such that, if the user requested a 5 second seek, they might be fine with up to 9 seconds of seek, but they wouldn't want to jump 20-30 seconds for some very keyframe-sparse video.

If I am understanding this code correctly it is a heuristic that bases its decision on the time taken by a single exact seek. The trouble with this is that the time taken for an exact seek depends upon where the position to seek to lands in relation to keyframes. If the position happens to land right on a keyframe then it is just as fast as a keyframe seek. This could happen with a video that has a large keyframe interval.

Yeah I thought that could be improved too. It would be better to at least sample a few seeks and average them together before making a firm conclusion. Or making a firmer conclusion based on the relative size of the delay, or some combination of collected stats.

[low-batt]

As mentioned by @krackers and discussed here, variable keyframe intervals is used by some codecs. Other codecs use a fixed interval. However in my opinion we should not put any work into an improved heuristic. Even a semi-reliable heuristic would still be confusing to users. We should focus on the point of view of the user as expressed by @saagarjha. They expect a seek of X seconds to seek X seconds.

I'm still thinking we need to effectively expose the mpv hr-seek option as I showed above and use yes as IINA's default value. We'd have to figure out how we want this setting to interact with the existing setting under Control.

[svobs]

I'm still thinking we need to effectively expose the mpv hr-seek option as I showed above and use yes as IINA's default value. We'd have to figure out how we want this setting to interact with the existing setting under Control.

I agree that using exact seeks should be the default and that should be the near-term focus. Did not mean to distract from that.

[low-batt]

We definitely can consider improving the heuristic, we just need to first figure out exactly how we are switching to using exact seeks by default. I will write up a specific proposal when I get a chance.

[saagarjha]

Do you have links to people who have asked for fast seeks in the past? Curious what their thoughts were.

[low-batt]

It is about performance when scrubbing. Scrubbing through a 4k AV1 video (no hardware decode) on my MacBookPro18,2 with the M1 Max chip by holding down the → key is fairly fast and responsive. If I enable exact seeking by setting hr-seek the change is very noticeable. Seeking is significantly slower.

There have been two changes that slowed down seeking and resulted in users complaining about seeking speed. One was a change to stop running the display link while playback was paused. The time to restart the display link was enough of a slow up to generate complaints. The other had to do with time to stop and start AirPods. Issues #3422 and #4153 are examples of such issues.

There is also issue #4215. That has to do with scrubbing backwards using exact seeks. To speed up that requires tuning mpv caches.

[Tinyu-Zhao]

I found that even changing it to "seek 5 exact" is of no avail

[low-batt]

That is expected to work, but does not due to the problem discussed in issue #3710. A fix for #3710 has been merged to the development version of IINA and will be included in the next release of IINA.

As a workaround, follow the instructions at the bottom of this comment from above to set hr-seek.