10 second latency when switching between subtitles

[crawfishie]

Bug report

Describe the bug

Here is a clear and concise description of what the problem is:

When switching between subtitles (internal, UTF-8 in a .mkv file) using the dialog box, "Next subtitle", or JSON, the current subtitles disappear immedately, but the newly selected ones take about 10 seconds to appear. This delay does not occur when toggling subtitles on from the off state, and I don't think it should be happening when switching between subtitles. The latency can be eliminated if I immediately do a seek, even a Seek(-1). However, Seek(0) does not help.

Expected Behavior

Here is a clear and concise description of what was expected to happen:

After switching from one subtitle to another, the newly selected ones were expected to appear within a second or two at most, just as quickly as they appear when subtitles are off, and you toggle them on.

Actual Behavior

It takes up to 10 seconds for the newly selected subtitles to appear after I switch between different subtitles.

Possible Fix

A seek not equal to zero causes subtitles to appear immediately. Perhaps this could be leveraged to address the problem without doing a consequential seek.

To Reproduce

Steps to reproduce the behavior:

1. Start playing a local file that contains more than one subtitle. For the attached log, my file contains two subtitles, one forced, the other default, both English. The forced subtitles are active when I start playing the file, which is expected and desired.
2. I pressed "L" for next subtitle, and the forced subtitles disappeared immediately. However, the default subs (which also contain the forced parts, as they should), were not displayed for about 10 seconds.
3. I used the Kodi subtitles dialog box to switch back to the forced subs. The current subs disappeared immediately, but the forced ones did not appear for about 10 seconds.
4. Finally, I used my enhanced toggle command, coded in Autohotkey and controlling Kodi through JSON, which switches between default and forced subtitles when forced ones are present. This differs from the Kodi toggle command, which would simply toggle the current subs on and off, which for this file, would by default mean toggling forced on and off, which does not do what I want. My JSON implementation incurs the same latency as the built-in Kodi features I used in (2) and (3).

Debuglog

The debuglog can be found here: https://paste.kodi.tv/xudoqazoju.kodi

Screenshots

Here are some links or screenshots to help explain the problem:

Additional context or screenshots (if appropriate)

Here is some additional context or explanation that might help:

I started a couple of threads about it here:

"Next subtitle" has 10 second latency before subtitles are shown: https://forum.kodi.tv/showthread.php?tid=365849

Why this matters to me so much: https://forum.kodi.tv/showthread.php?tid=365836&pid=3074425#pid3074425

In a nutshell:

1. There is no delay when toggling subtitles on/off.
2. When switching between subtitles, the current ones disappear immediately, but the newly selected ones take around 10 seconds to appear. This delay is eliminated by a non-zero Seek, even a Seek(-1).
3. In Autohotkey, I've implemented what I consider correct handling of the toggle command when forced subtitles are present, namely switching between default and forced, which is why this latency matters to me. The second post I linked to talks in more detail about what I mean by "correct handling", but that's really a separate issue and is not necessary to understand this report.

Your Environment

Used Operating system:

• Android

• iOS

• tvOS

• Linux

• OSX

• Windows

• Windows UWP

• Operating system version/name: Sony BRAVIA 4K VH2 with Android 10.0.0 API level 29, kernel: Linux ARM 32-bit version 4.19.75 (Also Windows 10 and Kodi 19.3 per the forum threads I created back in 12/21 and linked to above)

• Kodi version: 19.4 (19.4.0) Git:20220302-e12e66e019

note: Once the issue is made we require you to update it with new information or Kodi versions should that be required.
Team Kodi will consider your problem report however, we will not make any promises the problem will be solved.

[CastagnaIT]

please provide a log with Kodi 20 (happens also on kodi 20 ?)
Kodi 19 it is at end of development only important fixes will be backported

[FernetMenta]

@CastagnaIT check out #5312

[crawfishie]

@CastagnaIT check out #5312

Yep, sure sounds like my issue. Was it supposed to have been fixed back in 2014?

Also, I'm unclear about buffering being the cause. If I start playback in the middle of a forced subtitles scene, the subs appear immediately. There's no waiting 8 seconds as that thread talked about. Why can't switching between subtitles be as quick?

[matthuisman]

@crawfishie
i wonder if a little hacky kodi service add-on could help for now

eg.

class Player(xbmc.Player) :
    _last_sub=None
 
    def onAVStarted(self):
        self._last_sub = None
 
    def onAVChange(self):
        if self.isPlayingVideo():
            subtitle = self.getSubtitles()
            if subtitle and subtitle != self._last_sub:
                self._last_sub = subtitle
                self.seekTime(self.getTime()-1)

[crawfishie]

@crawfishie i wonder if a little hacky kodi service add-on could help for now

eg.

class Player(xbmc.Player) :
    _last_sub=None
 
    def onAVStarted(self):
        self._last_sub = None
 
    def onAVChange(self):
        if self.isPlayingVideo():
            subtitle = self.getSubtitles()
            if subtitle and subtitle != self._last_sub:
                self._last_sub = subtitle
                self.seekTime(self.getTime()-1)

That's a "consequential" seek I was hoping to avoid. For my own installation, I could do that in Autohotkey, where I've implemented a new "toggle subtitles" command that works more usefully when forced subs are present. More on that here:

https://forum.kodi.tv/showthread.php?tid=358553&pid=3003765#pid3003765

I don't do it because it would be rather intrusive, especially at the end of my "What did he say?" function. The start of the function is fine, because there's an implicit, expected Seek(-15), but after 20 seconds, an AHK timer fires, and subtitles are automatically turned off. It would be glitchy to do a Seek(-1) at the end of it to make forced subtitles appear immediately, which they should, because they're intended to be the "baseline" subtitles when present. OTOH, when forced subs are present, and subs are being switched instead of simply toggled, it's kind of perverse that my "What did he say?" function might leave you wondering "what he said" for 10 seconds after it's completed its duties, so maybe I will add a Seek(-1) to the end of it for that specific case, hopefully as a temporary measure.

[CastagnaIT]

from: #5312 (comment)

flushing only subs: this is what we had before -> you need to wait up to 8 seconds until subs a back

now I don't know how it works overall VideoPlayer.cpp is very extensive code
what I don't understand is why we have to wait for this buffering
cannot be indipendent the subtitle buffering from a/v?

[FernetMenta]

cannot be indipendent the subtitle buffering from a/v?

In general a video stream is comprised of video, audio, and subtitle streams. After demuxer the packets are progressively buffered by the stream players (up to 8 secs). The stream players have to synchronise output to VideoPlayer's clock. When buffers are full, demuxer is 8 secs ahead in time of output. Assume you switch subs from language A to language B. Since language B is not selected in he first place, those packets are thrown away right after demuxer. So right after switching the packets of language B have a timestamp of playing_time + 8 secs.
You can either seek back or buffer all languages for subtitles. btw: if you change the language of audio, a seek is done (like suggested by the PR). Obviously an inconsistency between how audio and subs are handled.

[mkreisl]

This problem has existed for ages, there have also been issues for it, but they have all obviously disappeared into oblivion.
And I have been using what @matthuisman suggested for a long time, in case I really need it.

Let's see if this will be fixed this time

[crawfishie]

You can either seek back or buffer all languages for subtitles.

FWIW, buffering just forced and default would be sufficient for my purposes. I only ever use my custom toggle and "What did he say?" commands, and I became aware of the delay issue only because they switch between forced and default when forced are present, which is how I think they should work.

I also take it that showing subs immediately when playback is started is fundamentally different than showing them immediately when switching from one to another during playback, and presumably the latter can't be made like the former without causing a noticeable audio/video glitch. Is this correct?

[matthuisman]

@CastagnaIT I couldn't see this same issue with Inputstream Adaptive streams on 19.4 windows. I suspect that's because they are handled differently?

[CastagnaIT]

@matthuisman you have to test with a MKV video the behaviour is same with all Kodi versions,
with ISAdaptive changes for webvtt should not influence this since the only change is that the sub packets are sent to the kodi directly as raw data instead of SRT packets chunks, in any case the kodi webvtt parser took very little time not seconds

its clear that VP need to be revisited in some way, kodi is the only player that need to wait the internal buffering to have subtitle displayed after a subs switching

if for some reason the internal buffer will be increased in the future, this will worsen this situation and many more users will complain, i noticed this problem a years ago and I never understood where the problem came from until now...

[FernetMenta]

Moving packet selection https://github.com/xbmc/xbmc/blob/master/xbmc/cores/VideoPlayer/VideoPlayer.cpp#L1611 to the stream players would solve the issue.

EDIT: updated link https://github.com/xbmc/xbmc/blob/Matrix/xbmc/cores/VideoPlayer/VideoPlayer.cpp#L1611

[crawfishie]

I've just played my first files with external .srt subtitles, one forced, the other not. Kodi Nexus Alpha switches between the two without delays. However, the files I described in my OP still incur the delay. Can anyone explain the discrepancy, why external text subtitles perform as desired, and internal text subtitles do not?

[crawfishie]

I've just played my first files with external .srt subtitles, one forced, the other not. Kodi Nexus Alpha switches between the two without delays. However, the files I described in my OP still incur the delay. Can anyone explain the discrepancy, why external text subtitles perform as desired, and internal text subtitles do not?

Bump for no reply in 2.5 months. Anyone?

[CastagnaIT]

@crawfishie the problem is about buffered packages in VP

there is a kind of propose to try make a fix in the comment above,
but currently we have few developers per VP area, may be involved in other tasks

I have too little VP knowledge, i am not able to make the changes myself

[pannal]

Hey, this is still an issue in Omega, is there anything planned for this? Thanks!