-
Notifications
You must be signed in to change notification settings - Fork 57
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Seekable differs from non-MSE behavior #5
Comments
As a consequence of the current seekable definition and step 8 of the video seeking algorithm, I believe out-of-buffer seeking is not possible for live streams. MSE specifies the seekable ranges to be the union of the current buffered regions and the video spec constrains seeking to time positions within the seekable range. To make things a bit more concrete, imagine you have a sliding-window style live stream manifest with five video segments currently available. After a bit of buffering, your player has downloaded one segment. In Safari and iOS devices, you would be allowed to set the current time to any position that is greater than three segment durations from the latest available position. With Media Source Extensions, you would only be able to seek within the segment you currently have buffered:
|
The issue in this bug seems to involve consistency of behavior, but not interoperability, assuming I understand it correctly. The impact seems to limit how close to real-time a user can seek on live streams. Moving beyond the limits of buffered data should temporarily stall playback until buffering is restored, while limiting the seek to within the buffered range would minimize the risk of that. |
Yes, the issue is about consistency with the HTML spec's guidelines for live playback behavior. I may be missing something in your point about real-time playback. It seems to me that the stream can be joined arbitrarily close to real-time by adjusting where to begin appending data into a Source Buffer. What you can't do (I think) is instruct the video element to seek back two minutes if you only have one currently appended in your Source Buffer, unlike media with a known duration or non-MSE live implementations. |
It sounds like we need a mechanism similar to With non-live content the app can set With live content the MediaSource can't look at duration because it must be Infinity, and can't assume the start time is zero. So it currently only relies on So it seems like it would helpful if the MediaSource provided more control over defining the start time and end time of the available media timeline. What if it were direct control over what seekable returns?
Changes to those properties could reuse the |
@heff that's a great summary of the problem, thanks. Something like your proposal does seem like it would address the issue. One minor suggestion: Time Ranges support non-contiguous regions. I don't know of a streaming format that makes use of that feature but it could be added without complicating the API too much since Time Ranges are normalized: mediaSource.addSeekableRange(240, 360);
mediaSource.removeSeekableRange(260, 280); |
If I understand correctly, there are two live-stream seeking problems related to MSE described so far in this issue:
Both of these could lead to non-ideal default UI controls and behavior, especially when combined. @dmlap I am confused, though, by "What you can't do (I think) is instruct the video element to seek back two minutes if you only have one currently appended in your Source Buffer, unlike media with a known duration or non-MSE live implementations." |
@wolenetz I'd agree with your formulation of the issues. I didn't realize the start point of One follow-up question on the start point of |
The issue of SourceBuffer running out of space is orhogonal, IMHO, and handled separately by the user agent's implementation of the coded frame eviction algorithm (http://www.w3.org/TR/media-source/#sourcebuffer-coded-frame-eviction). In appendBuffer() usage, this algorithm is run prior to parsing newly appended media into the timeline (and throws a QuotaExceededErr if not enough space was freed). In appendStream() usage, the same behavior occurs synchronously during the call of appendStream(); the eviction algorithm also runs during each iteration of the stream append loop, though no QuotaExceededErr is thrown from there, but there is an error transition by which an app can discover the problem. Regardless, if the user-agent's choice of removal ranges within its coded frame eviction algorithm was insufficient to free enough space, the app is given notification and can explicitly Remove() buffered ranges. The problem really is two-fold:
I believe an API like MediaSource.setSeekableRange([start,end]) could satisfy both problems (in #2, an app could achieve more interoperable behavior by explicitly setting a non-finite seekable range that overrides the default logic in existing MSE spec. jdsmith@, is this new API suggestion acceptable? I think it solves at least one real problem users of MSE are having, which is "how can apps reliably control the MSE seekable range in live (infinite duration) streams?" |
@jdsmith3000 have you been able to give this issue some thought? I'd be happy to attempt a PR against the spec with @wolenetz's suggestion if that would help move things along. |
Would this API then alter the response from mediaSource.seekable? And would the effects persist as streaming continued? @wolenetz: Given the double limitation you site on the seekable TimeRange, what does Chrome return for a live stream with duration = infinity? To me, it makes sense to return the full range, and have the app then limit the forward seeking to current real time. |
For consistency with the video element's seekable attribute, I think the API should override the response of |
I've had some discussions here, but haven't closed. We aren't convinced that having the app set a seekable range is the right solution. It seems instead that we might want a concept where apps can jump to the live edge. Some formats might also have problems with zero based timestamps (e.g. MPEG-2 TS timestamps roll over every 26 ½ hours, so it’s not as simple as taking the timestamp for a segment and mapping it into a simple zero based timestamp). |
Could you elaborate on what sort of concept you had in mind to allow an app to jump to the live edge? The only ideas I can come up with seem to require info from the app about where the live edge is, which devolves into a |
+interop label to follow-up on the TimeRanges discrepancy in Edge vs others (also, need to confirm if unrestricted double TimeRanges are being done in Edge) |
@jdsmith3000, assuming that by this you mean that end times in The per-spec model for seeking media elements is that the requested time is clamped to seekable ranges, and then normally that's precisely where you'll end up. If at all possible, I think MSE should behave the same way. Simply allowing the seekable ranges to be set sounds pretty good to me. If you'd like a constructor for |
I'm aware of live streaming players built on the current API that accomplish seekable ranges by implementing custom controls and their own time ranges. These span from the live edge back through a DVR window maintained by the server. The current API isn't specifically aware of either limit, and knows only that duration is +infinity. Allowing apps to set a seekable range for this case could approximate both limits and enable live seeking using either the standard controls or custom ones that still use the seekable range from the API. Presumably the seekable range would be set at least with each append of new sourceBuffer data. That append contains the most current live data, and the app can extrapolate the DVR limit using window data from the manifest or some other source. We previously talked about these times persisting once set. That makes seekeable limits in this case the responsibility of the app, and that's probably fine. We would likely want to somehow limit apps from using this on VoD content, where valid time ranges exist. This might be done by allowing seekable ranges to be set only on duration +infinity content, or where the API itelf cannot supply valid time ranges. Assuming we can agree on this limitation, I'd support going ahead with implementing something like this approach:
I'm less clear on the proposed |
Completing my previous statement: I'm less clear about the proposed |
For seekable ranges with gaps in them, and would like to discuss this at the TPAC session. |
I don't recall the TPAC discussions for sure, but believe that there was some concern about letting a custom range clip the buffered one. I think your suggestion of using the union of the custom range and the buffered one addresses this, and means that custom ranges are primarily for expanding seekable ranges, likely into live DVR time ranges. Is it your intent that setting custom ranges can be done multiple times, and individual ranges are stored for use in calculating the union with buffered data? I've been thinking we want a single custom range to for DVR type uses, and this would need to be updated frequently to insure the oldest DVR time is accurate. On naming, I think shorter is better, and prefer setSeekableRange despite the union operation with buffered data. |
@wolenetz @jdsmith3000 anything I can do to help resolve this issue? Sounds like the preference is for a seekable range setter instead of get/remove. #42 doesn't fit that bill but if it's roughly the sort of contribution you'd expect, I'd be happy to trying a new patch for |
@dmlap, your PR is currently pending on:
|
@wolenetz thanks for the update. I don't know if or when I'll be able to resolve (1). If my PR is making this issue harder to address, please don't hesitate to close it. I'll be happy knowing this live streaming use-case is supported by MSE v1-- it doesn't matter to me if my wording is used or not. |
OK, it looks like we need to make progress on this. I'll compose a PR soon to attempt to fix this. |
@wolenetz: When is your proposed PR going to be ready for review? |
We synced on this during today's editors' sync. @jdsmith3000 and I are on the same page and I will proceed with a PR. The naming of the new method is still something we'd like to simplify, and we also think just a single range (if any) for the custom_seekable_range is necessary (we don't have any other use cases identified which need multiple disjoint custom_seekable_ranges, and the existing spec behavior for seekable with finite duration is a single range [0,duration). |
fyi - this is a test message, since I'm suddenly getting a warning on all my github pages: I've contacted their support. Hopefully this won't impact my immediate work. |
I didn't unassign myself. Looks like GH's "mostly harmless robot" did that and I can't reassign myself. I'll continue until blocked... |
It looks like GH has now just fixed my profile problem. That outage happened at a bad time, to say the least. |
Fix #5: Add support for customizing seekable on live MSE streams
…gerald See w3c/media-source#5 MozReview-Commit-ID: EP37gRmUAXF --HG-- extra : rebase_source : cd134517a3f1d50bce2f17a82acabe0f2bb042cb
…gerald See w3c/media-source#5 MozReview-Commit-ID: EP37gRmUAXF
…gerald See w3c/media-source#5 MozReview-Commit-ID: EP37gRmUAXF
…gerald See w3c/media-source#5 MozReview-Commit-ID: EP37gRmUAXF UltraBlame original commit: fcb130529e8206fb65ed0bff9837762d29cd7a1a
…gerald See w3c/media-source#5 MozReview-Commit-ID: EP37gRmUAXF UltraBlame original commit: fcb130529e8206fb65ed0bff9837762d29cd7a1a
…gerald See w3c/media-source#5 MozReview-Commit-ID: EP37gRmUAXF UltraBlame original commit: fcb130529e8206fb65ed0bff9837762d29cd7a1a
I am not sure if this is a 100% related but how are we seeking when we load audio files in chunks or segments? The issue I am having is that if I want to play an mp3 from the middle, See my stackoverflow question for details. Any help on this would be awesome! |
@falk-stefan, I've responded to your stackoverflow question today. Also, in addition to duration, you'll probably want to inspect in your repro what your media element reports as its buffered ranges when the seek is issued. (e.g. element.buffered). |
The MSE spec seems to indicate that the highest end time for seekable should never exceed the highest buffered time when the duration is set to Infinity. The HTML standard indicates that duration should be Infinity for unbounded or live media and that user agents should be very liberal determining seekable ranges for media.
Safari on iOS and OSX seems to have interpreted this as meaning that the seekable range should include the time ranges covered by all segments in the current "sliding window" of content in a live HLS video. That definition is convenient because it makes seeking to the live point or building a DVR interface a simple operation for downstream developers, and seems in keeping with the spirit of the HTML standard. It does not seem possible to configure Source Buffers or a Media Source to achieve the same effect. Is there a mechanism to override seekable with out-of-band info like you might get from an M3U8?
The text was updated successfully, but these errors were encountered: