blacklist the playlist that has stopped being updated and never blacklist the final playlist available by zhuangs · Pull Request #1039 · videojs/videojs-contrib-hls

zhuangs · 2017-03-06T18:53:32Z

Description

Detect that playlist has stopped being updated in a timely manner and blacklist the playlist
Never blacklist the final playlist available #1026

mjneil · 2017-03-07T19:45:35Z

          addSeekableRange();
        }
+
+        buffered = this.tech_.buffered();


I think it would be good to pull out this section of calculations to a separate function as this event handler is getting quite gnarly

changed this

mjneil · 2017-03-07T19:49:13Z

+        // and the buffered data ends at the end of the last segment in the playlist
+        if (bufferedTime <= Ranges.TIME_FUDGE_FACTOR &&
+            endTime <= Ranges.TIME_FUDGE_FACTOR &&
+            currentTime > seekableEnd) {


seekableEnd, lastBufferedEnd and endTime have the potential of being NaN at this point since seekableEnd and lastBufferedEnd may never be assigned a value if the conditionals for the if statements fail, and playEnd may be null from this.playlistEnd() affecting the calculation of endTime

I thought seekableEnd, lastBufferedEnd, endTime being NaN may only happen at the very beginning of a playlist and that isn't the case we should blacklistCurrentPlaylist, is this correct? In that case, it doesn't affect the if statement. Did you mean we should check seekableEnd, lastBufferedEnd, endTime isNaN() inside the if?

With lastBufferedEnd for example, you declare lastBufferedEnd at the start of this function, so it is undefined. The only location lastBufferedEnd is assigned a value is on line 378, which is inside the if (buffered.length) check. If buffered.length === 0 then lastBufferedEnd will still be undefined and any calculation involving lastBufferedEnd e.g. endTime = playEnd - lastBufferedEnd will result in NaN.

With that said, looking at this again, it may be ok for this to happen. Since any comparison with NaN will return false, seekable.length === 0 or buffered.length === 0 probably means we don't have enough information to know if the playlist stopped updating and probably shouldn't blacklist it. This will require testing to see if we can create a scenario in which those values result in NaN but we still want to blacklist

So I was giving this more thought and I think we can avoid worrying about it all together by calling this method and blacklisting the playlist outside of the loadedplaylist listener. Playlist Loader already knows when a playlist it has loaded didn't actually update the live window. The isPlaylistOutdated_ method more so checks if the player is stuck at the end despite loading "fresh" playlists. If instead playlist loader triggered an event at that moment it knows the playlist isnt a fresh update, then MasterPlaylistController can listen to that and run the checks for playlistOutdated and if those checks pass, then we blacklist. This way we wont run these checks every time the playlist is refreshing correctly and run the risk of accidentally blacklisting a playlist we don't want to

changed as suggested

mjneil · 2017-03-07T19:56:27Z


    this.seekable_ = videojs.createTimeRanges();
    this.hasPlayed_ = () => false;
+    this.playlistEnd = playlistEnd;


It isn't necessary to attach this function to MasterPlaylistController, you can just call playlistEnd directly where you need it. On top of that, if you attach playlistEnd to the default export of playlist.js, the default export is already included in the Hls object available in this file. If you go that route, you do not need to import this function in a seperate import call, and instead could just call Hls.Playlist.playlistEnd()

I attached this function to MasterPlaylistController because I was trying to call this function in the videojs-contrib-hls.test.js file. but maybe there is better way to do this

Ah didn't notice that. Hmmmm, its not a big deal to attach the function, but if you come up with another solution that doesnt require it, thatd be great

yes, I found directly override Hls.Playlist.playlistEnd() in the test file is better :)

mjneil · 2017-03-07T19:57:47Z

+ * @returns {Object} an object containing the time length of the playlist
+ * @function playlistEnd
+ */
+


this function is very similar to seekable defined just below. Is there any way we can reduce the amount of repeat code here?

changed this

mjneil · 2017-03-07T20:11:24Z

Would be good to have a some tests for playlistEnd

mjneil · 2017-03-13T17:23:43Z


    return segmentSync.time - sumDurations(playlist, syncIndex, 0);
  }
+  return 0;


This should return null by default isntead of 0. If there isn't enough information to determine expired time, returning 0 would be sending false information if there actually is expired time.

Did this in the first place but then I found there is a case that the playlist contains the endList, so the seekableEnd will return the duration of playlist but the seekableStart returns null. I think that's why I changed here to return 0 and check for null on the sync points instead of checking expired.

That is a good point, but I don't think returning 0 is the right call here since there are live scenarios where we may not have any sync points, but there has been expired time, we just can't calculate it. Since endList indicates a VOD asset, we actually already have a known sync point then just off knowing endList alone, which would be { time: 0, mediaSequence: 0 } since VOD always starts at time 0 segment 0. I'd say it would make most sense to update getPlaylistSyncPoints to set expiredSync to { time: 0, mediaSequence: 0 } if playlist.endList, otherwise set to playlist.syncInfo, and finally resorting to null. That way calculateExpiredTime can properly calculate for VOD case, while still returning null in the live case where we don't have enough information

mjneil · 2017-03-13T17:25:05Z

+  if (playlist.endList) {
+    return duration(playlist);
+  }
+  let { expiredSync, segmentSync } = getPlaylistSyncPoints(playlist);


Since you moved getPlaylistSyncPoints to calculateExpiredTime calling it again here just to check for null on expiredSync segmentSync is unnecessary work. If calculateExpiredTime starts returning null when it can't calculate expired, you can just check for null on expired instead of checking the sync points

mjneil · 2017-03-13T17:52:53Z

+   * Check if a playlist has stopped being updated
+   * @param {Object} playlist the media playlist object
+   * @return {boolean} whether the playlist has stopped being updated or not
+   */


minor, but I think we can come up with a better name for the method. Maybe isPlaylistOutdated_, isPlaylistStagnant_, isPlaylistUpdating or something along these lines that is a bit more descriptive of what the method does. (note the return value should reflect the name logically). This method should also be private, indicated by the _ appended to the end of the name

…seekable and playlistEnd to reduce the amount of repeat code and add tests for playlistEnd

… checking the sync points and rename playlist update check function

mjneil · 2017-03-15T22:31:48Z

      });
    });
+
+    this.masterPlaylistLoader_.on('playlistnotupdate', () => {


maybe name the event playlistunchanged ?

mjneil · 2017-03-15T22:43:58Z

+   * Check if a playlist has stopped being updated
+   * @param {Object} playlist the media playlist object
+   * @return {boolean} whether the playlist has stopped being updated or not
+   */


I know I've suggested renaming this already, but now that we have an event that signifies the playlist is outdated/not refreshing, and this method more checks if we are stuck at the end of the playlist, so maybe somthing like stuckAtPlaylistEnd_or isAtPlaylistEnd_. Then our logic sort of semantically reads like if the refreshed playlist is unchanged and the player is at the end of the playlist, blacklist this playlist.

makes sense! renamed!

mjneil · 2017-03-16T14:50:00Z

+                           '16.ts\n');
+  // trigger a refresh
+  this.clock.tick(10 * 1000);
+


I think it would be good to add a check between the two refreshes to make sure the playlist wasnt blacklisted early. (you may have to move the play and playing triggers up, not sure though)

…l rendition

…rt loading after errors

…nd add check for if lastRemovedSegment is null

gesinger

Would be good to have some tests for stuckAtPlaylistEnd_ directly.

gesinger · 2017-03-17T18:16:15Z

+    }
+    bufferedTime = lastBufferedEnd - currentTime;
+
+    // the end of the current playlist


extra comment

gesinger · 2017-03-17T18:17:17Z

+    let currentTime;
+    let playEnd;
+    let endTime;
+


Can remove a lot of lines up top if we create the variables where they are first used.

gesinger · 2017-03-17T18:22:36Z

+
+    // the time between end of the playlist and the end of the buffered
+    endTime = playEnd - lastBufferedEnd;
+


If there is no seekable range, can we be technically stuck at playlist end?

gesinger · 2017-03-17T18:27:41Z

+    bufferedTime = lastBufferedEnd - currentTime;
+
+    // the end of the current playlist
+    // playEnd = this.playlistEnd(playlist);


Since this is only used in one place, it may be clearer to just use it in the calculation directly: endTime = Hls.Playlist.playlistEnd(playlist) - lastBufferedEnd; . We should also change the comment to say it is using the absolute end of the playlist, and not the safe live end.

gesinger · 2017-03-17T18:54:56Z

+
+    if (buffered.length) {
+      lastBufferedEnd = buffered.end(buffered.length - 1);
+    }


While NaN can sometimes be OK, it is usually good to try to avoid using it. In this case, where we need the amount of time buffered, this should be 0 instead of NaN.

gesinger · 2017-03-17T19:26:45Z

+ */
+
+export const playlistEnd = function(playlist) {
+  return calculatePlaylistEnd_(playlist);


If this function just calls the other function, do we need both?

gesinger · 2017-03-17T19:44:18Z

+  return null;
+};
+
+const calculatePlaylistEnd_ = function(playlist, useSafeLiveEnd) {


Would be good to keep some documentation about the method, particularly the definition of useSafeLiveEnd (i.e., "live playlists should not expose three segment durations worth...")

gesinger · 2017-03-17T20:00:22Z

+    return null;
+  }
+  endSequence = useSafeLiveEnd ? Math.max(0, playlist.segments.length - Playlist.UNSAFE_LIVE_SEGMENTS) : Math.max(0, playlist.segments.length);
+  let end = intervalDuration(playlist,


Can just return directly instead of creating a variable for it.

gesinger · 2017-03-17T20:10:02Z

+  openMediaSource(this.player, this.clock);
+  this.player.tech_.triggerReady();
+
+  this.requests[0].respond(200, null,


We might be able to use standardXHRResponse to save some lines. We can also use this.requests.shift to make it so that the latest outstanding request is always at the beginning of the array (no need to keep track of them via indexing).

gesinger · 2017-03-17T20:10:56Z

+  url = this.requests[1].url.slice(this.requests[1].url.lastIndexOf('/') + 1);
+  media = this.player.tech_.hls.playlists.master.playlists[url];
+
+  assert.ok(!media.excludeUntil, 'playlist didnt be blacklisted');


playlist wasn't blacklisted

gesinger · 2017-03-23T20:00:49Z

-      videojs.log.warn('Problem encountered with the current ' +
-                       'HLS playlist. Switching to another playlist.');
+      videojs.log.warn('Problem encountered with the current HLS playlist.' +
+                 (error.message ? ' ' + error.message : '') +


Indentation

changed! sorry for keeping making this mistake

…ne media playlist

mjneil · 2017-03-24T20:28:26Z

+  this.player.tech_.setCurrentTime(50);
+  this.player.tech_.buffered = () => videojs.createTimeRange();
+  Hls.Playlist.playlistEnd = () => 130;
+  assert.ok(!this.masterPlaylistController.stuckAtPlaylistEnd_(playlist), 'not stuck at playlist end');


Since stuckAtPlaylistEnd_ only uses seekable to check if we have a seekable range, but doesn't use seekable.end(0) for any of its checks, is there even a point to checking not stuck at playlist end when currentTime not at seekable end even if the buffer is empty?

If we still want to keep this part in, I'm not sure having playlistEnd and seekable.end(0) both be the same (130) is representative of an actual scenario since seekable.end is the time since playback started to minus 3 segment durations from the end of the window, where playlistEnd is time since playback started to minus 0 segment durations from the end of the window.

Good point. I think that leaving in tests that make use of seekable values is good, even if stuckAtPlaylistEnd_ doesn't use them, since the tests shouldn't know too much about the internal workings of the function. But you're right that the scenario of playlistEnd equaling seekable's end shouldn't be a common case. However, in the event that a playlist changes from LIVE to VOD (at the end of a stream), and the only update was the addition of #EXT-X-ENDLIST, seekable's end should equal playlistEnd, and I think it's possible that this case is a valid one.

mjneil · 2017-03-24T21:09:55Z

-
-  let expiredSync = playlist.syncInfo || null;
-
+  let expiredSync = playlist.endList ? { time: 0, mediaSequence: 0} : playlist.syncInfo || null;


When a live stream is finished, usually an EXT-X-ENDLIST tag is applied to the end of the playlist even though it wasn't originally a VOD. This would set expiredSync to { time: 0, mediaSequence: 0 } even if that isn't an actual sync point. We should prioritize playlist.syncInfo if it exists, otherwise { time: 0, mediaSequence: 0 } if playlist.endList is true, otherwise null.

zhuangs force-pushed the fix/blacklisting-stuck-playlist branch from 61b4072 to 3d28543 Compare March 6, 2017 19:46

blacklist the playlist that has stopped being updated

b3b9598

zhuangs force-pushed the fix/blacklisting-stuck-playlist branch from 3d28543 to b3b9598 Compare March 7, 2017 15:14

mjneil reviewed Mar 7, 2017

View reviewed changes

mjneil self-assigned this Mar 7, 2017

mjneil reviewed Mar 13, 2017

View reviewed changes

Shan Zhuang added 2 commits March 13, 2017 15:27

pull out the playlistEnd calculation to a separate function, rewrite …

cee3d10

…seekable and playlistEnd to reduce the amount of repeat code and add tests for playlistEnd

handle the expired for VOD case, check for null on expired instead of…

5fe7303

… checking the sync points and rename playlist update check function

zhuangs force-pushed the fix/blacklisting-stuck-playlist branch from 978bef5 to 5fe7303 Compare March 13, 2017 19:27

add event listener for playlist isn't a fresh update in playlist-loader

fa12f4f

zhuangs force-pushed the fix/blacklisting-stuck-playlist branch from 400b990 to fa12f4f Compare March 15, 2017 18:51

mjneil reviewed Mar 15, 2017

View reviewed changes

rename event and function

e11859d

mjneil reviewed Mar 16, 2017

View reviewed changes

Shan Zhuang and others added 7 commits March 16, 2017 13:01

changes on test cases

cb3a2a0

never blacklist final available final rendition

406718b

simplify finalRendition check function

ea0654b

reset the loader state after error occurs to keep requesting the fina…

8773591

…l rendition

delay the requests for final rendition and get the segment loader sta…

4e1721b

…rt loading after errors

clear mediaUpdateTimeout after isFinalRendition block in load function

f9c0d35

bind a call to load in setTimeout, return after setting the timeout a…

7022dff

…nd add check for if lastRemovedSegment is null

gesinger reviewed Mar 17, 2017

View reviewed changes

zhuangs force-pushed the fix/blacklisting-stuck-playlist branch from 73af052 to 68f4687 Compare March 20, 2017 20:19

Shan Zhuang added 2 commits March 23, 2017 10:42

add test for playliststuck and warning log message

458baec

change log message for not blacklisted final rendition and other changes

5b1b5fd

gesinger reviewed Mar 23, 2017

View reviewed changes

indentation

d5dbf4c

gesinger approved these changes Mar 23, 2017

View reviewed changes

Shan Zhuang and others added 6 commits March 23, 2017 16:17

indentation

2beba1f

add test case to verify playlist fails fast when there is only that o…

c6ef7e3

…ne media playlist

change test name and add check for error before request

59a8a5b

add test cases

5f3fa67

modify test name

7d5e582

minor change on test case

ba5e071

mjneil reviewed Mar 24, 2017

View reviewed changes

mjneil added tested and removed needs:testing labels Mar 24, 2017

zhuangs force-pushed the fix/blacklisting-stuck-playlist branch from bab9852 to d5dbf4c Compare March 24, 2017 20:42

mjneil reviewed Mar 24, 2017

View reviewed changes

prioritize syncInfo if it exists

5b179a5

mjneil approved these changes Mar 24, 2017

View reviewed changes

Shan Zhuang added 2 commits March 24, 2017 17:46

solve conflict

eccd669

resolve test conflicts

a79bdf7

zhuangs force-pushed the fix/blacklisting-stuck-playlist branch from f62c7c9 to a79bdf7 Compare March 24, 2017 22:01

mjneil added needs:testing and removed tested labels Mar 24, 2017

zhuangs changed the title ~~blacklist the playlist that has stopped being updated~~ blacklist the playlist that has stopped being updated and never blacklist the final playlist available Mar 24, 2017

mjneil mentioned this pull request Mar 24, 2017

Never blacklist the final rendition available #1026

Closed

mjneil approved these changes Mar 24, 2017

View reviewed changes

mjneil added tested and removed needs:testing labels Mar 24, 2017

gesinger merged commit aeceda3 into videojs:master Mar 24, 2017

gesinger mentioned this pull request Apr 26, 2017

Fix for #550 and #555 #561

Closed


		// the time between end of the playlist and the end of the buffered
		endTime = playEnd - lastBufferedEnd;


		let expiredSync = playlist.syncInfo \|\| null;

		let expiredSync = playlist.endList ? { time: 0, mediaSequence: 0} : playlist.syncInfo \|\| null;

Conversation

zhuangs commented Mar 6, 2017 • edited by mjneil Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mjneil commented Mar 7, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zhuangs Mar 13, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gesinger left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zhuangs commented Mar 6, 2017 •

edited by mjneil

Loading

zhuangs Mar 13, 2017 •

edited

Loading