Skip to content

Comments

Automatically run webgpu tests if enable_experimental_webgpu is enabled#1569

Merged
fhanau merged 1 commit intomainfrom
felix/gpu-auto-tests
Feb 9, 2024
Merged

Automatically run webgpu tests if enable_experimental_webgpu is enabled#1569
fhanau merged 1 commit intomainfrom
felix/gpu-auto-tests

Conversation

@fhanau
Copy link
Contributor

@fhanau fhanau commented Jan 23, 2024

The WebGPU tests were previously not included in the bazel build system. With this change we have 5 more tests that will run on CI and achieve better code coverage.

@fhanau fhanau requested a review from edevil January 23, 2024 15:20
@fhanau fhanau requested review from a team as code owners January 23, 2024 15:20
@fhanau fhanau requested review from dom96 and mikea January 23, 2024 15:20
@fhanau
Copy link
Contributor Author

fhanau commented Jan 23, 2024

@edevil Looks like all tests are failing on Linux and one is failing on macOS. I was hoping these would work out of the box since all tests are passing locally on mac... If you can take a look at the Linux issues we can probably enable all/some of these.

@edevil
Copy link
Contributor

edevil commented Jan 23, 2024

I haven't looked at the mac one yet, but the linux ones fail because we couldn't find a compatible Vulkan driver installed in the system:

Warning: vkCreateInstance failed with VK_ERROR_INCOMPATIBLE_DRIVER

Are there any CI Linux images that have a working Vulkan installation? In my Linux dev VM I can use a software-only rasterizer via LLVMpipe and mesa-vulkan-drivers.

@fhanau
Copy link
Contributor Author

fhanau commented Jan 23, 2024

Vulkan

Looks like this is possible, but perhaps prohibitively slow: actions/runner-images#2998
I can probably add a needs-webgpu tag here that would allow us to only run this on CI on Windows for now, so we can run the tests on at least some CI jobs.

@edevil
Copy link
Contributor

edevil commented Jan 23, 2024

The mac failure is a bit weird. We render an image to a texture, copy it to a buffer, hash the content and we compare the hash value with a known good value previously calculated in Chrome.

The hash value is correct in my Linux VM (vulkan), which uses LLVMpipe, and in my Mac (metal) laptop that uses the M2 chip. It also seems to be calculated correctly in the windows (D3D) CI. But in the MacOS (metal) CI it is using a virtual GPU "Apple Paravirtual device" and seems to calculate the wrong value. I'll have to see if I can somehow simulate that environment in my laptop but whatever it is should take some time to debug.

@edevil
Copy link
Contributor

edevil commented Jan 23, 2024

Looks like this is possible, but perhaps prohibitively slow: [actions/runner-images#2998](https://github.com/actions

Do you mean slow to install the tools, or slow to run the gpu workload? Because this test workload is very simple, it should run fast even on emulated GPUs.

Copy link
Contributor

@dom96 dom96 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! lgtm but based on the discussion we probably want to update this to only run on Windows for now

@fhanau fhanau force-pushed the felix/gpu-auto-tests branch from 2575285 to f771490 Compare February 2, 2024 13:46
@fhanau
Copy link
Contributor Author

fhanau commented Feb 9, 2024

I ended up amending the PR to use the off-by-default tag – this way it is easy to run the tests individually or as a whole while they won't run by default. Unfortunately, there was no elegant way to make this run only on Windows CI and still make it easy to run locally on macOS/Windows. If there's more cases like these where we have tests that can't/shouldn't run in CI but work well locally we can revisit this.

@fhanau fhanau merged commit 06c09e7 into main Feb 9, 2024
@fhanau fhanau deleted the felix/gpu-auto-tests branch February 9, 2024 15:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants