How to actually run wgpu on CI?

[almarklein]

It would be very nice if we could run tests with wgpu on CI, also in pygfx, to better validate our code. Actually, I would be a bit disappointing if we can't.

Considerations

GPU vs integrated vs software

I briefly looked into GPU instances (e.g. at Scaleway), but these are aimed at really high performance work and therefore really expensive. Strictly speaking we don't need a GPU, as long as Vulkan/Metal has something to run on (Intel graphics or software), but I'm not sure how that'd work in a VM.

Possible approaches (generic)

Vulkan on CI

One way for this to work is Vulkan/Metal to be available on the CI machines, which does not appear to be the case by default (see #45), as could have been expected. There are some posts suggestion that other projects setup their own local server via e.g. Gitlab runner.

DirextX

DX11/DX12 is available on GH Actions, but it is prone to crashing (for our examples).

WGPU

In theory there could be a software WGPU implementation that does not rely on hardware.

Relevant links

• Run WebGPU CTS on CI gfx-rs/wgpu#1421 - more or less same issue on wgpu
• https://github.com/kazan-3d/kazan
• https://swiftshader.googlesource.com/SwiftShader/
• https://www.phoronix.com/scan.php?page=news_item&px=Lavapipe-Teases-Vulkan-1.1

last edit: june 2021

[Korijn]

OpenGL works on CI, so I think vulkan etc should too... We can install drivers if need be?

[almarklein]

Isn't OpenGL on CI usually a software driver? So maybe ... something like https://github.com/kazan-3d/kazan?

[Korijn]

Yeah, that is an option.

But I also think that Azure Pipelines agents do have Intel Graphics available.

We've used https://marketplace.visualstudio.com/items?itemName=ms-autotest.screen-resolution-utility-task in the past to run selenium tests for a certain WebGL based application you are very familiar with. ;) It enables a screen session. I'm not sure if that would help us here, but at least it's evidence that OpenGL works on Azure Pipelines CI machines. Also; we didn't need to install a software driver there, but then again, maybe it was preinstalled... hmm

[Korijn]

I've looked through gfx-rs/wgpu and they don't run this type of tests on Travis CI either. Not sure if this is feasible at all

[Korijn]

We could indeed try to build kazan and use it to run tests on CI. But that would only work on Linux since it is Vulkan, I think?

[almarklein]

But that would only work on Linux since it is Vulkan, I think?

AFAIK it can run on any platform on which it can be compiled. Windows 10 also includes Vulkan, I believe. However, I don't know how wgpu-native (or glx-hal) is going to pick it up. Kazan seems very much bleeding edge, so I'd rather try other things first :)

[Korijn]

With some helpful explanations from @kvark I think we've cracked this one: gfx-rs/wgpu#644

Just need to compile swiftshader (and point wgpu-native to it).

[Korijn]

Sadly I'm not able to get swiftshader to compile: it's poorly documented which versions and configurations of build toolchains should be used and there are no git tags of "releases" that are supposed to compile successfully either. I don't have time to reverse engineer all that info.

I noticed that the latest version of OSMesa actually has a Vulkan API as well so I will see if I can get that to work instead!

cc @kvark

[kvark]

@Korijn it's a bit sad state of things that Swiftshader doesn't feature any binary releases at all. They have a Vulkan ICD interface defined already, so they could do backwards-compatible releases for everyone to enjoy. Perhaps, you could file a bug to https://issuetracker.google.com/issues?q=componentid:408190 ?

[Korijn]

There is code to build Vulkan mesa, and apparently Doom Eternal can run on it, so I'm hopeful we can use it for our purposes here as well:

https://github.com/mesa3d/mesa/tree/master/.gitlab-ci

But it's a little hard to extract the core steps to make a build ourselves...

[kvark]

I wonder what this Vulkan mesa is? The only software implementation I thought existed was SwiftShader.

[Korijn]

Looks like they're using lavapipe now:

https://github.com/gfx-rs/wgpu/blob/b679342d9e62e50eff307d785590ffcbcc39b0d8/.github/workflows/ci.yml#L111

Wonder if that could be an easy solution for us, too!

Alternatively, it might be time to try DirectX 12 again