Consider relaxing timing of initial HAVE_METADATA transitioning #275

wolenetz · 2021-05-22T00:19:27Z

Some implementations, including Chromium, have a media pipeline on distinct thread(s)/process(es) versus the web apps' thread(s). Delaying the relevant updateend associated with the appendBuffer() whose parsing reaches the "set the HTMLMediaElement.readyState attribute to HAVE_METADATA part of step 7 of https://www.w3.org/TR/media-source/#sourcebuffer-init-segment-received until that step is fully completed is not done in some implementations, so the application can buffer more media faster while the pipeline is still transitioning in parallel.

This issue tracks standardizing some allowance for this behavior, as allowing improved application buffering agility is probably a good idea.

-editted: The delay in Chromium is slightly less than I had thought (decoder setup doesn't necessarily block initial transition to HAVE_METADATA, but the metadata notification from MSE to the media element still hops threads through the pipeline.)

The text was updated successfully, but these errors were encountered:

Unless the app explicitly sets `mediaSource.duration`, `HTMLMediaElement.duration` remains NaN until initial HAVE_METADATA is reached, even if the attached mediaSource has already buffered media well beyond the initialization segment(s) necessary to begin transition to HAVE_METADATA. In Chromium, that transition is begun asynchronously, letting thread hop through the pipeline thread complete while letting the app continue. Eventually, the media element transitions, but in the interim, its value for duration could still be NaN, even if mediaSource has a duration value. This change relaxes the changeType-play-* utility's reliance on strict matching of mediaElement.duration and mediaSource.duration, instead relying on the latter for use in trimming the buffered duration. I've filed MSE spec issue #275 to discuss this behavior: w3c/media-source#275 This change also reports more details in changeType web-test assertion failures, which also enabled finding the suspected root cause of the tests' flakiness. Bug: 1184745 Change-Id: I208cbfbbc60776366a16b6a3e79f52480df5be37

Unless the app explicitly sets `mediaSource.duration`, `HTMLMediaElement.duration` remains NaN until initial HAVE_METADATA is reached, even if the attached mediaSource has already buffered media well beyond the initialization segment(s) necessary to begin transition to HAVE_METADATA. In Chromium, that transition is begun asynchronously, letting thread hop through the pipeline thread complete while letting the app continue. Eventually, the media element transitions, but in the interim, its value for duration could still be NaN, even if mediaSource has a duration value. This change relaxes the changeType-play-* utility's reliance on strict matching of mediaElement.duration and mediaSource.duration, instead relying on the latter for use in trimming the buffered duration. I've filed MSE spec issue #275 to discuss this behavior: w3c/media-source#275 This change also reports more details in changeType web-test assertion failures, which also enabled finding the suspected root cause of the tests' flakiness. Bug: 1184745 Change-Id: I208cbfbbc60776366a16b6a3e79f52480df5be37 Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2906776 Auto-Submit: Matthew Wolenetz <wolenetz@chromium.org> Reviewed-by: Ted Meyer <tmathmeyer@chromium.org> Commit-Queue: Ted Meyer <tmathmeyer@chromium.org> Cr-Commit-Position: refs/heads/master@{#886148}

wolenetz · 2021-05-25T18:39:06Z

Note that with MSE-in-Workers (#175), information like duration and track metadata may propagate to the HTMLMediaElement at slower rate than atomically during initialization segment received algorithm (running in a MediaSource Worker), by nature of the algorithm running in a different context than the main window. So this relaxation of legacy main thread element duration == main thread mediasource duration requirement wouldn't conflict with MSE-in-Workers specification/expectations of delay in main thread media element duration eventually matching what worker-thread mediasource duration was.

…ilures, a=testonly Automatic update from web-platform-tests MSE: Fix rare flaky changeType-play-* failures Unless the app explicitly sets `mediaSource.duration`, `HTMLMediaElement.duration` remains NaN until initial HAVE_METADATA is reached, even if the attached mediaSource has already buffered media well beyond the initialization segment(s) necessary to begin transition to HAVE_METADATA. In Chromium, that transition is begun asynchronously, letting thread hop through the pipeline thread complete while letting the app continue. Eventually, the media element transitions, but in the interim, its value for duration could still be NaN, even if mediaSource has a duration value. This change relaxes the changeType-play-* utility's reliance on strict matching of mediaElement.duration and mediaSource.duration, instead relying on the latter for use in trimming the buffered duration. I've filed MSE spec issue #275 to discuss this behavior: w3c/media-source#275 This change also reports more details in changeType web-test assertion failures, which also enabled finding the suspected root cause of the tests' flakiness. Bug: 1184745 Change-Id: I208cbfbbc60776366a16b6a3e79f52480df5be37 Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2906776 Auto-Submit: Matthew Wolenetz <wolenetz@chromium.org> Reviewed-by: Ted Meyer <tmathmeyer@chromium.org> Commit-Queue: Ted Meyer <tmathmeyer@chromium.org> Cr-Commit-Position: refs/heads/master@{#886148} -- wpt-commits: 04c1391243b470ae1e0a0b9a7a53280c9fa66e57 wpt-pr: 29052

wolenetz · 2021-09-03T18:03:17Z

Requesting this be discussed at Sept 14 Media WG meeting.

This change relaxes the timing expectation for 'updateend' of an appended initialization segment meaning the media element must already have reached HAVE_METADATA. In Chrome, the transition to media element readyState hops threads and may occur after the SourceBuffer's 'updateend' has been dispatched. See the following MSE spec issue for discussion of this behavior: w3c/media-source#275 The test now awaits both 'updateend' and 'loadedmetadata' receipts before issuing a load() that is expected to close the MediaSource. This change also removes an older copy of the same test, updating the wpt version and related test expectations. BUG=1143720,641121 Change-Id: I87df990e3a032c6f7dadf25829093c35913c7855

This change relaxes the timing expectation for 'updateend' of an appended initialization segment meaning the media element must already have reached HAVE_METADATA. In Chrome, the transition to media element readyState hops threads and may occur after the SourceBuffer's 'updateend' has been dispatched. See the following MSE spec issue for discussion of this behavior: w3c/media-source#275 The test now awaits both 'updateend' and 'loadedmetadata' receipts before issuing a load() that is expected to close the MediaSource. This change also removes an older copy of the same test, updating the wpt version and related test expectations. BUG=1143720,641121 Change-Id: I87df990e3a032c6f7dadf25829093c35913c7855 Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3450339 Auto-Submit: Matthew Wolenetz <wolenetz@chromium.org> Reviewed-by: Ted (Chromium) Meyer <tmathmeyer@chromium.org> Commit-Queue: Ted (Chromium) Meyer <tmathmeyer@chromium.org> Cr-Commit-Position: refs/heads/main@{#969124}

…es, a=testonly Automatic update from web-platform-tests MSE: Fix flaky mediasource-detach failures This change relaxes the timing expectation for 'updateend' of an appended initialization segment meaning the media element must already have reached HAVE_METADATA. In Chrome, the transition to media element readyState hops threads and may occur after the SourceBuffer's 'updateend' has been dispatched. See the following MSE spec issue for discussion of this behavior: w3c/media-source#275 The test now awaits both 'updateend' and 'loadedmetadata' receipts before issuing a load() that is expected to close the MediaSource. This change also removes an older copy of the same test, updating the wpt version and related test expectations. BUG=1143720,641121 Change-Id: I87df990e3a032c6f7dadf25829093c35913c7855 Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3450339 Auto-Submit: Matthew Wolenetz <wolenetz@chromium.org> Reviewed-by: Ted (Chromium) Meyer <tmathmeyer@chromium.org> Commit-Queue: Ted (Chromium) Meyer <tmathmeyer@chromium.org> Cr-Commit-Position: refs/heads/main@{#969124} -- wpt-commits: 6fa7cbfe3d493a6d6799da8d2af91ab71c489f84 wpt-pr: 32780

This change relaxes the timing expectation for 'updateend' of an appended initialization segment meaning the media element must already have reached HAVE_METADATA. In Chrome, the transition to media element readyState hops threads and may occur after the SourceBuffer's 'updateend' has been dispatched. See the following MSE spec issue for discussion of this behavior: w3c/media-source#275 The test now awaits both 'updateend' and 'loadedmetadata' receipts before issuing a load() that is expected to close the MediaSource. This change also removes an older copy of the same test, updating the wpt version and related test expectations. BUG=1143720,641121 Change-Id: I87df990e3a032c6f7dadf25829093c35913c7855 Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3450339 Auto-Submit: Matthew Wolenetz <wolenetz@chromium.org> Reviewed-by: Ted (Chromium) Meyer <tmathmeyer@chromium.org> Commit-Queue: Ted (Chromium) Meyer <tmathmeyer@chromium.org> Cr-Commit-Position: refs/heads/main@{#969124}

…es, a=testonly Automatic update from web-platform-tests MSE: Fix flaky mediasource-detach failures This change relaxes the timing expectation for 'updateend' of an appended initialization segment meaning the media element must already have reached HAVE_METADATA. In Chrome, the transition to media element readyState hops threads and may occur after the SourceBuffer's 'updateend' has been dispatched. See the following MSE spec issue for discussion of this behavior: w3c/media-source#275 The test now awaits both 'updateend' and 'loadedmetadata' receipts before issuing a load() that is expected to close the MediaSource. This change also removes an older copy of the same test, updating the wpt version and related test expectations. BUG=1143720,641121 Change-Id: I87df990e3a032c6f7dadf25829093c35913c7855 Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3450339 Auto-Submit: Matthew Wolenetz <wolenetz@chromium.org> Reviewed-by: Ted (Chromium) Meyer <tmathmeyer@chromium.org> Commit-Queue: Ted (Chromium) Meyer <tmathmeyer@chromium.org> Cr-Commit-Position: refs/heads/main@{#969124} -- wpt-commits: 6fa7cbfe3d493a6d6799da8d2af91ab71c489f84 wpt-pr: 32780

wolenetz · 2022-09-16T20:40:29Z

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Consider relaxing timing of initial HAVE_METADATA transitioning #275

Consider relaxing timing of initial HAVE_METADATA transitioning #275

wolenetz commented May 22, 2021 •

edited

Loading

wolenetz commented May 25, 2021

wolenetz commented Sep 3, 2021

wolenetz commented Sep 16, 2022

chrisn commented Nov 8, 2022

chrisn commented Dec 7, 2022

Consider relaxing timing of initial HAVE_METADATA transitioning #275

Consider relaxing timing of initial HAVE_METADATA transitioning #275

Comments

wolenetz commented May 22, 2021 • edited Loading

wolenetz commented May 25, 2021

wolenetz commented Sep 3, 2021

wolenetz commented Sep 16, 2022

chrisn commented Nov 8, 2022

chrisn commented Dec 7, 2022

wolenetz commented May 22, 2021 •

edited

Loading