Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DRM related tests race on network latency #4135

Open
SingingTree opened this issue Oct 31, 2016 · 5 comments
Open

DRM related tests race on network latency #4135

SingingTree opened this issue Oct 31, 2016 · 5 comments

Comments

@SingingTree
Copy link
Contributor

SingingTree commented Oct 31, 2016

The DRM tests have network dependencies that result in races. Independent of if the tests are accessed locally or at http://w3c-test.org/ they need to make requests to the license server, so there is always some variable lag due to this. However, even in relatively straight forward tests, such as drm-events this appears to result in rare intermittent fails, even when running locally (for my case a license request must round tripped from New Zealand).

I've been looking at adjusting some of the test timeouts, but it's a bit of a whack a mole. I'm raising this issue so further discussion can be had around how to best handle this.

@cpearce
Copy link
Contributor

cpearce commented Oct 31, 2016

I suggest all .drm. tests have their timeout extended across the board. Then we don't need to keep playing whack-a-mole and up the timeouts as we trip over intermittent failures caused by latency with the DRM license server.

@ddorwin
Copy link
Contributor

ddorwin commented Oct 31, 2016

@mwatson2, any thoughts?

@ddorwin ddorwin modified the milestone: V1 Oct 31, 2016
@jdsmith3000
Copy link
Contributor

I guess the question is: Are they reasons to limit these tests to 10 second execution limits? The only downside might be a bit longer execution time on valid timeout results. @cpearce; Do you have an alternate time in mind (e.g. 20s)?

@ddorwin
Copy link
Contributor

ddorwin commented Nov 1, 2016

Yes, that would be my main concern. Looking at the latest results, we don't have many timeouts, so I think this should be rare and thus okay. (We want tests to fail rather than timeout if possible. See #3938. In some cases, such as depending on an event, it is not possible.)

@SingingTree or @cpearce, please prepare a PR.

@mwatson2
Copy link
Contributor

mwatson2 commented Nov 1, 2016

The test harness offers two choices of timeout: short (10s) or long (60s). We'll need to add the following to every HTML file:

<meta name="timeout" content="long">

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants