Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clean up proprietary codec checks/expectations in LayoutTests #9288

Merged
merged 1 commit into from Feb 2, 2018

Conversation

Projects
None yet
4 participants
@chromium-wpt-export-bot
Copy link
Collaborator

chromium-wpt-export-bot commented Jan 30, 2018

This is mostly a reland of
https://chromium-review.googlesource.com/c/chromium/src/+/807604
with some improvements and now unblocked by changes to bots.

Media layout tests need repair

  1. Many tests check for proprietary codec support by only asking about
    "video/mp4" without specifying a codec. This check is busted
    because MP4 is no longer considered proprietary while codecs like
    H264 and AAC still are. We currently claim "video/mp4" is entirely
    unsupported via test hacks to avoid breaking those checks.
  2. Some MSE tests focus on MP4 with H264. While many bots now have the
    needed codecs, we artificially remove their support via test-only
    hacks in order to keep expectations the same for all runners.

This CL does the following

  1. Remove incomplete checks for "video/mp4" - instead hardcode use of
    the .ogv files (preserving the existing behavior).
  2. Update MSE mp4/h264 test expectations to PASS|FAIL. This gives us
    some coverage on bots with proprietary codecs while allowing failure
    on bots that lack codecs.
  3. Remove test hacks that artificially reduce codec support.

It is also not a goal to make LayoutTests try all the supported codecs.
That is covered by unit/integration tests.

Due to 3) above, some external/wpt/media-source tests now fail because
they select mp4. Fixing these failures is tracked in Issue 794338.

Bug: 327115,746579,787575,568704,794338,807110
Change-Id: I05565d7ca9d8d5b12391785888701723810671d5
Reviewed-on: https://chromium-review.googlesource.com/849641
Commit-Queue: Chrome Cunningham chcunningham@chromium.org
Reviewed-by: Dale Curtis dalecurtis@chromium.org
Reviewed-by: Peter Beverloo peter@chromium.org
Reviewed-by: Matthew Wolenetz wolenetz@chromium.org
Cr-Commit-Position: refs/heads/master@{#533924}

@wpt-pr-bot
Copy link
Collaborator

wpt-pr-bot left a comment

Already reviewed downstream.

@chromium-wpt-export-bot chromium-wpt-export-bot force-pushed the chromium-export-cl-849641 branch from 9276ae6 to e414bdf Jan 30, 2018

@w3c-bots

This comment has been minimized.

Copy link

w3c-bots commented Jan 30, 2018

Build PASSED

Started: 2018-02-02 02:43:15
Finished: 2018-02-02 02:47:06

Failing Jobs

  • safari:11.0
  • chrome:dev

Unstable Results

Browser: "Chrome Dev" (failures allowed)

View in: WPT PR Status | TravisCI

Test Subtest Results Messages
/media-source/mediasource-endofstream.html   TIMEOUT: 2
OK: 8
  MediaSource.endOfStream(): duration and buffered range end time before and after endOfStream FAIL: 10
assert_equals: SegmentInfo duration should still roughly match mediaSource duration expected 6.548 but got 6.549
  MediaSource.endOfStream(): duration truncated to 0 when there are no buffered coded frames FAIL: 10
assert_equals: expected 0 but got 2
  MediaSource.endOfStream(): media element notified that it now has all of the media data TIMEOUT: 2
PASS: 8
Test timed out

Browser: "Safari 11.0" (failures allowed)

View in: WPT PR Status | TravisCI

Test Subtest Results Messages
/media-source/mediasource-endofstream.html   TIMEOUT: 1
OK: 9
  MediaSource.endOfStream(): duration and buffered range end time before and after endOfStream FAIL: 9
assert_equals: SegmentInfo duration should still roughly match mediaSource duration expected 6.548 but got 6.549
  MediaSource.endOfStream(): duration truncated to 0 when there are no buffered coded frames PASS: 9
  MediaSource.endOfStream(): media element notified that it now has all of the media data PASS: 9

@chromium-wpt-export-bot chromium-wpt-export-bot force-pushed the chromium-export-cl-849641 branch from e414bdf to 6cffcd1 Feb 1, 2018

Clean up proprietary codec checks/expectations in LayoutTests
This is mostly a reland of
https://chromium-review.googlesource.com/c/chromium/src/+/807604
with some improvements and now unblocked by changes to bots.

Media layout tests need repair
1) Many tests check for proprietary codec support by only asking about
   "video/mp4" without specifying a codec. This check is busted
   because MP4 is no longer considered proprietary while codecs like
   H264 and AAC still are. We currently claim "video/mp4" is entirely
   unsupported via test hacks to avoid breaking those checks.
2) Some MSE tests focus on MP4 with H264. While many bots now have the
   needed codecs, we artificially remove their support via test-only
   hacks in order to keep expectations the same for all runners.

This CL does the following
1) Remove incomplete checks for "video/mp4" - instead hardcode use of
   the .ogv files (preserving the existing behavior).
2) Update MSE mp4/h264 test expectations to PASS|FAIL. This gives us
   some coverage on bots with proprietary codecs while allowing failure
   on bots that lack codecs.
3) Remove test hacks that artificially reduce codec support.

It is also not a goal to make LayoutTests try all the supported codecs.
That is covered by unit/integration tests.

Due to 3) above, some external/wpt/media-source tests now fail because
they select mp4. Fixing these failures is tracked in Issue 794338.

Bug: 327115,746579,787575,568704,794338,807110
Change-Id: I05565d7ca9d8d5b12391785888701723810671d5
Reviewed-on: https://chromium-review.googlesource.com/849641
Commit-Queue: Chrome Cunningham <chcunningham@chromium.org>
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
Reviewed-by: Peter Beverloo <peter@chromium.org>
Reviewed-by: Matthew Wolenetz <wolenetz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#533924}

@chromium-wpt-export-bot chromium-wpt-export-bot force-pushed the chromium-export-cl-849641 branch from 6cffcd1 to ac170b2 Feb 2, 2018

@chromium-wpt-export-bot chromium-wpt-export-bot merged commit 4c16ce5 into master Feb 2, 2018

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

@chromium-wpt-export-bot chromium-wpt-export-bot deleted the chromium-export-cl-849641 branch Feb 2, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.