Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run WebGPU CTS on CI #1421

Closed
kvark opened this issue Jun 1, 2021 · 3 comments · Fixed by #1903
Closed

Run WebGPU CTS on CI #1421

kvark opened this issue Jun 1, 2021 · 3 comments · Fixed by #1903
Assignees
Labels
area: infrastructure Testing, building, coordinating issues

Comments

@kvark
Copy link
Member

kvark commented Jun 1, 2021

Is your feature request related to a problem? Please describe.
We need to have the strongest CI coverage on this repo if we want it to live independent.

Describe the solution you'd like
My first guess is integrating with Deno. Obviously, this will only work for non-breaking changes.
It could be OK, given the expectation that most changes in the future are non-breaking.

Describe alternatives you've considered
If that doesn't work out, we can also try Servo and Gecko.

Additional context
This is a requirement for moving move things into wgpu repo. It was born out of discussion in gfx-rs/gfx#3768, followed by an internal meeting at Mozilla.

@kvark kvark added the area: infrastructure Testing, building, coordinating issues label Jun 1, 2021
@kvark kvark pinned this issue Jun 1, 2021
@kvark kvark added this to the Family Reunion milestone Jun 1, 2021
@kvark
Copy link
Member Author

kvark commented Jun 1, 2021

Our friends at Deno provided this example https://github.com/lucacasonato/wgpu_cts_runner
Edit: CTS modifications to get going - gpuweb/cts@03edd5a

@kvark
Copy link
Member Author

kvark commented Jun 18, 2021

This isn't actually a requirement for the milestone, but still a very important issue to get sorted out.

@kvark kvark removed this from the Family Reunion milestone Jun 18, 2021
@kvark
Copy link
Member Author

kvark commented Aug 25, 2021

#1859 brings the Deno bindings in-repo. Let's consider this option (1).
I think it's an important decision, so trying to see what other options we have:

(2) Deno_webgpu could live in a separate crate. If there is a major PR here that we want to be safe about, we'd have to ask the author to also have a branch that updates the bindings. This is cumbersome, it's easy to fail following this.

(3) Deno_webgpu could generate the API traces for all of the tests and store them somewhere, to be picked up by CI. This would still be problematic to test if the format of an API trace changes.

(4) Keep relying on manual CTS runs. This is fragile.


The costs of integrating Deno:

  • quite a bit more code to maintain, especially considering that things like texture formats are repeated multiple times now
  • much larger Cargo.lock, which is arguably can be moved out of the repo
  • dependency on Deno project
  • much increased CI times (this is the natural outcome of any of 1-3)

I think the costs are justified: Deno team has been pro-active in adopting WebGPU and helping us with CTS. And the benefit of a consistent CI coverage is worth it, especially now that PR would be able to unlock more tests. So let's do it!

@kvark kvark self-assigned this Sep 3, 2021
@kvark kvark unpinned this issue Sep 10, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area: infrastructure Testing, building, coordinating issues
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant