Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

split variants in WebGPU's cts.html for less timouts #30999

Open
sagudev opened this issue Jan 3, 2024 · 3 comments
Open

split variants in WebGPU's cts.html for less timouts #30999

sagudev opened this issue Jan 3, 2024 · 3 comments
Labels
A-content/webgpu The WebGPU implementation.

Comments

@sagudev
Copy link
Member

sagudev commented Jan 3, 2024

We need more fine grained solution either via splitting test or adding timeout only to specific tests (or both).

OLD:
I though I catch all [FAIL/PASS, TIMEOUT, NOTRUN], but apparently not. To prevent them from happening it would be bast to have variants for flaky tests.

See: gpuweb/cts#3233

@sagudev sagudev added the A-content/webgpu The WebGPU implementation. label Jan 3, 2024
@sagudev sagudev changed the title split varints cts.html for every subtest split variants in WebGPU's cts.html for every subtest Jan 3, 2024
@sagudev sagudev changed the title split variants in WebGPU's cts.html for every subtest split variants in WebGPU's cts.html for flaky test Jan 15, 2024
@sagudev
Copy link
Member Author

sagudev commented Jan 15, 2024

I am currently waiting for gpuweb/cts#3246 to land.

@sagudev
Copy link
Member Author

sagudev commented Mar 21, 2024

Currently this was resolved (somehow) in #31695, by just disabling such flaky test. But now in #31772 I observed some real timeouts that would be solved by more fine chunking.

Solution as noted in #31772 (comment), is to chunk only problematic test per max time.

@sagudev
Copy link
Member Author

sagudev commented Mar 28, 2024

Times from Chrome are completely different (servo/FF: 18s vs chrome 20ms), so we would need to generate our own time data for it to work (can we share data with firefox).

Moving to timeout long produced this results (from current main): https://github.com/sagudev/servo/actions/runs/8456974386/job/23169503338.

In the future we can move all test that need more time in separate html file so meta long will effect only them.

@sagudev sagudev changed the title split variants in WebGPU's cts.html for flaky test split variants in WebGPU's cts.html for less timouts Apr 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-content/webgpu The WebGPU implementation.
Projects
Status: To do
Development

No branches or pull requests

1 participant