Ray Tracing extension #535
Comments
|
See #477 which is almost a duplicate of this issue. The work is nice and shows will be a good reference if/when we add ray-tracing to WebGPU but as mentioned in the other issue we don't have enough perspective on this technology yet to be able to add it to WebGPU. Also I don't see us looking at this space in the medium term because there's already so much to do for basic functionality :) |
|
Adding ray tracing to the web platform in a way it can be implemented using Metal Performance Shaders, D3D12 ray tracing, and Vulkan ray tracing, seems like a valuable thing to do. I agree with @Kangz about priorities. If someone wants to do the work for us, I’m sure we’d be willing to accept it, but short of that, the WebGPU team on WebKit (at least) is busy with the existing WebGPU spec and functionality. I do expect we’ll get there one day eventually, though. |
|
Note that not many desktop titles have added support for VKRay nor DXR yet. It remains to be seen whether it will see large-scale adoption. DXR 1.1 is a recent upgrade to the DXR API which makes it (in my opinion) a lot more palatable for developers. Not sure if equivalent functionality exists in VKRay. |
|
@maierfelix the steps to proceed here would be:
|
|
@kvark thanks for the direction, but could you elaborate about number 2? Edit: Do you mean the different render backends? |
|
|
Vulkan does this with |
|
I think it would be a major win for WebGPU to include ray tracing support (optional based on hw support), or in the very least have it on the near term road-map. Would love to avoid the 'mistake' of WebGL 2, which mostly matched GLES 3, while GLES 3.2 added all the fun stuff (geometry, tessalation and compute shaders). This really made WebGL 2 seem as a very disappointing update when it came out (I know it still has lots of good stuff compared to WebGL 1 - but still) :S |
|
For your interest, I've added a specification of the RT draft here. DirectX and Vulkan Ray Tracing are very similar in their style and usage.
My current concerns are:
|
|
Hello! I am Web 3D engine developer! Very Thanks for the work of @maierfelix ! I wish WebGPU to support Ray-Tracing extension as soon as possible! Thanks for @maierfelix and WebGPU!!! |
|
Seems like there now is standard vulkan ray tracing extension: https://www.khronos.org/news/press/khronos-group-releases-vulkan-ray-tracing VK_KHR_ray_tracing (https://www.khronos.org/registry/vulkan/specs/1.2-extensions/html/vkspec.html#VK_KHR_ray_tracing) |
|
I've made an example of the VK_KHR_ray_tracing extension here, which I'll use as a base to port the current VK_NV_ray_tracing implementation in Dawn. Currently I'm mainly waiting for bug fixes related to the current NVIDIA driver, the validation layers don't support the new extension and it's not possible to build AS on the host yet. The new extension indeed offers the same features as DXR, though most of them are not implemented in the drivers yet. Regarding Metal and OpenGL: |
|
@maierfelix Great work! Thank you very much! I wish node webgpu could support the new extension~ |
|
@yyc-git The RT fork now uses VK_KHR_ray_tracing and is also included in the latest version of node-webgpu. If you want to try it, make sure you have the Vulkan beta driver installed. |
|
@magcius Great work! Thank you very much~ |
|
Great work, using the official KHR extension is a way going forward. After the new Metal4(possibly) releases next month WWDC, I hope someone who familiar with Apple MPS raytracing to contribute to the metal backend, or give some advices to shape the spec to make the metal implementation more straightforward. |
|
The RT fork now supports Ray tracing through the D3D12 backend. VK_KHR_ray_tracing and DXR are almost identical APIs, except 2 small differences I've found so far. DXR requires the maximum ray payload size and maximum hit attribute size defined during the pipeline creation. In VK_KHR_ray_tracing this is not necessary, but possible to define too. |
|
Just want to say, I think adding raytracing makes sense, it seems to be the way forward with graphics. AMD, NVIDIA and INTEL are all adding hardware support for raytracing, with much of it been available this year. There are even fallback layers for older hardware / some current gen. Also Apple's Metal now has a raytracing API I can only speculate but I imagine mobile RT hardware for Android will become a thing here soon too, there are some companies working on RT hardware for Android, such as Imagination Technologies. I believe Samsung's new Arm SoC designs are using AMD new RDNA2 and I'd be really surprised if they omitted support for VKR/RT. @maierfelix Thanks for working on this! |
|
Making this official will be valuable to enable real-time high-quality visualization for web-based applications. |
|
Please note: shipping ray-tracing is not going to be a priority for the group until well after 1.0 has shipped and bugs have been ironed out, because it is a feature which is only available to a small fraction of users with recent high end machines. |
|
For the record, I think it's totally fine for a sub-group to form that would develop this part of the API as an extension, as long as it doesn't distract the main group from moving forward. I'd be happy to participate, knowing that there is a lot of demand for RT! |
|
I’m interested in working on ray tracing too |
|
Metal-style and DXR-style raytracing are such different beasts that it will be extremely difficult to see a convergence any time soon. Not to mention that DXR has two different versions, 1.0 and 1.1, but VKray is roughly equivalent to DXR 1.0. There's far too much divergence in the space to make a long-lasting web standard, in my opinion. DXR has still not seen large-scale adoption, and game developers are still skeptical of the promises made by DXR 1.0 and 1.1. Namely, there's still a lot of questionable "performance portability" and best practices, especially given the lack of ray sorting. |
|
I suppose another issue to figure out would be de-noising. Seems like with current hardware, this is really needed to reach a good framerate (with decent quality). You could probably just let that be up to the application, but would be easier if there was some support for this available directly. |
Happy to hear that!
The current beta extension VK_KHR_ray_tracing targets at matching DXR's functionality, there are quite a few new features in the recent releases of both APIs but most features are not that relevant to essentially work with ray tracing (Ray queries for example). The approach of adding ray tracing as a separate pipeline (just like a compute pipeline) is in my opinion the best choice here, and is in fact how I designed the API in my Dawn RT fork and what is also the initial design of VK_NV_ray_tracing. Metal is indeed a problem as it doesn't follow the principles of the other APIs. I'm afraid that if this issue is tackled, Metal as the weakest unit in this regard either limits the way of how DXR/VK_KHR_ray_tracing can be used too much, or even becomes a full blocker.
I have seen a very different picture here. There are many people looking forward to working with RT hardware and the people that do are often surprised of how fast it is. A remarkable part of people lacks the motivation to go through the hassle of dealing with these low-level APIs in order to work with hardware RT. This is where I see WebGPU filling this gap. I built numerous projects with the RT fork and it was super fun and you get nice results within hours instead of weeks or months. Regarding performance, the current issue is the lack of a broad documentation, best practice documents being sparely and just typical issues that rise with new stuff, I don't see this too much as a problem. Also ray sorting is a general problem in ray tracing along many others.
Implementing a denoiser should be up to the user, denoisers are too specialized and too much tied to an application and also too much of a research field. |
I agree, I'd love to use ray tracing on the web. I think web browsers would be a great way to pioneer (as in popularize through widespread use and demos) this relatively new feature through an extension. People did amazing stuff just with fragment shaders in shader toy, perhabs we'd also get so see amazing things done with ray tracing in a similarly easy to use platform. E.g. using compute to generate procedural models and scenes, and ray tracing to render them. |
|
Re: denoising: yeah, maybe best handled by libraries. Just thinking about new users wanting to play with ray tracing, and perhaps being a bit disappointed by quality/performance. So if denoising library(ies) were available to easily plug in to your app, that could be a nice solution. Anyway can be handled outside the scope of a main RT extension. |
|
Just to echo what @kvark said, ray tracing is exciting and all, but the focus should be on making a first version of WebGPU available. Without that first version there can be no ray tracing extension and we shouldn't slow down development of core WebGPU for experimental ray-tracing work. (also I have concerns that it might be too early to investigate ray-tracing because that space isn't super mature in native yet) |
|
@yyc-git that looks really cool, 加油 |
Yes, I have implement BMFR denoising based on your WebGPU Node project and BMFR's code in my real-time hybrid ray tracing demo. The detail is here: WebGPU+光线追踪Ray Tracing 开发三个月总结 But still my BMFR denoising has some defects, e.g. the specular highlight denoised result is not smooth! @pjoe Thanks~ |
|
I've taken the liberty of creating a matrix room for further discussion: https://app.element.io/#/room/#webgpu-ray-tracing:matrix.org @yyc-git Great writeup on zhihu.com, though had to use google translate (我的中文不好) |
|
@pjoe Thanks for your room~ |
|
I've made some progress implementing ray-tracing support into wgpu: gfx-rs/wgpu#3020. You can see what APIs I've added here: https://github.com/gfx-rs/wgpu/pull/3020/files#diff-797d05eb01678dbe95794b18b0271e468af2cf1e5df45031f08289a157940c12 I haven't used ray-tracing extensively so I'm not 100% sure on what we should be exposing. A notable difference to the API that @maierfelix created is that:
I'd love to be able to collaborate and see what makes the most sense for a generic API over both DXR and Vulkan RT! |
|
Nice work! If I remember correctly, when starting with I've never really looked into the query-based RT APIs, but to me it looks like this would make the emulation of VKR/DXR in Metal much harder or up to impossible. Please correct me if I'm wrong on this. An uncertain but very important question is still how APIs without hardware RT support (such as Metal) should be tackled. Questions like if the RT extension should simply be unavailable on these machines, or if there should be some kind of emulation. Personally I'd vote against emulation, since such emulation could result in heavy limitations for the hardware APIs and make the general implementation a lot more complicated. This would require to emulate the RT implementations of GPU vendors on driver-level, which I think is a tremendous task, especially when formalising a standard. Also as previous people mentioned in this issue, collecting the API differences between VKR and DXR would be incredibly useful in formalising a proper API proposal and determining the overall scope of such an extension. |
I didn't think about security at all with the code I've written so far. Currently I've added 3 new buffer uses: const BUFFER_DEVICE_ADDRESS = 1 << 10;
const BOTTOM_LEVEL_ACCELERATION_STRUCTURE_INPUT = 1 << 11;
const TOP_LEVEL_ACCELERATION_STRUCTURE_INPUT = 1 << 12;With the scratch buffer requiring I'll do this in the PR soon. As far as Metal goes.. it sounds like it'd be hard to unify their ray tracing API with Vulkan RT/DXR no matter what we do. But I'm happy to hear otherwise from people more experienced with it. |
|
Very nice getting some ray-tracing in WGPU! (even if it is a draft PR) I'm not super knowledgeable about ray-tracing but the wgpu PR seems to only add the ability to create the acceleration structures and perform ray queries in the shader. But my understanding is that to make ray-tracing scale to full-featured engines, you need to be able to dispatch a shader depending on the triangle that's hit / not hit etc. Which requires shader tables and a lot more complexity. In general to standardize features we do a fairly detailed investigation of the feature in all three APIs (D3D12, Metal, Vulkan) and shading languages, highlighting common points and differences. Then we find some design that would work on all of them and start from there. In this case it seems Metal only has a performance shader that does ray-intersection against an accelerator structure and that's it? It might be difficult to merge all APIs. (but maybe possible by combining with other Metal features to do bindless shader selection, not sure). Expect standardization of ray-tracing to be a very long process though (and a process that can only start after v1 so as to not distract the group). It may be possible and useful to prototype experimental extensions in wgpu or Dawn before that though. |
Apparently, Metal added proper full ray tracing API according to: https://twitter.com/gavkar/status/1275673056845991937. |
I haven't looked into it much, but I think that ray queries should suffice for things like ray-traced shadows and reflections. GPU path tracers that do things like tracing more complex objects/proceural objects such as spheres will probably want the full pipeline though. For performance info on a ray-tracing pipeline vs ray queries see https://tellusim.com/rt-perf/.
|
Another takeaway from this document is that a custom implementation on Metal is slightly faster than the native metal code! So if we wanted to, we could skip the Metal intefaces for acceleration structures and build our own BVH on the GPU:
Still, I think it's worth looking into metal ray tracing. I'm happy to sacrifice a bit of acceleration structure build performance if it means we get native metal support, but I'm not willing to sacrifice Vulkan/DX12 trace performance. |
|
The tellusim rt-perf benchmark is difficult to talk about without source code behind it, because many of these RT APIs have large pitfalls where you can fall off the performance fast path easily, in subtle ways, especially because of fundamental vendor differences. You could interpret it as some "average" app coded in maybe obvious ways with these ray-tracing APIs, though I suspect the variance would be meaningfully large, and it's hard to know where in the ranges these results sit. |
|
Metal's ray tracing API is a first-class citizen in Metal: https://developer.apple.com/documentation/metalperformanceshaders/metal_for_accelerating_ray_tracing?language=objc We have custom intersection shaders that can perform ray-triangle intersection (e.g. for things like alpha-tested geometry) and ray-AABB intersection (e.g. for procedural 3D geometry). As @Kangz described above, there is a relationship between the items in the acceleration structure and which intersection shaders get run. The intersection shaders rely on bindless workflows. The author can attach a small number (compared to the total number of resources) of argument buffers to the intersection shaders. These argument buffers essentially act as the root(s) of a tree where the leaves point to all the resources which could be used by the intersection shader. The code within the shader is responsible for navigating this tree. If WebGPU created a ray tracing extension, it should definitely be implementable on top of Metal's ray tracing facilities. |
|
Another comment: right now the set of addressable resources in a WebGPU draw/dispatch is not enough to support ray tracing, as the ray could travel anywhere in the whole scene. Before WebGPU supports ray tracing, we need to solve this problem (presumably by shipping a bindless extension). |
|
Hi everyone, really great to see here the interests in WebGPU ray tracing! I just open sourced my project WebRTX (https://github.com/codedhead/webrtx) which extends WebGPU API with ray tracing capabilities. The interface is mostly based on @maierfelix 's dawn ray tracing spec but it's software ray tracing. Basically I have implemented compute shader code that does the BVH traversal and Shader Binding Table based user shader function invocation etc., and this code is combined with user provided GLSL ray tracing shaders into one final mega compute shader. I have a website https://shadowray.gl for the demo. The project was done almost two years ago, hopefully I will have some time to work on it again. |
Well Done! Good Job! So now we can use ray tracing pipeline with WebGPU in Chrome browser based on your webrtx project, and in the fureture the same code can run on hardware RTX when WebGPU support rtx ray tracing pipeline! Thanks for your contribution! |
Thanks! Yes exactly, I started the project as a polyfill implementation for the potential WebGPU ray tracing extension. Although the implementation is far from complete in the current state. |
|
@codedhead really nice work! |
|
Bump. Im making a game engine on the web using WebGPU, and while it certanly runs, its not even close to fast. Adding this Ray-Tracing extension will make AABB and Triangle tests much faster, making my engine actually runnable. |
|
Hardware ray tracing will not be available in WebGPU for a good while, I'd say no less than 1-2 years unfortunately. If you'd like your engine to work well sooner than that, you'll need to find ways to use what's available now or is already in development. |
|
I think that the presence of ray tracing is inevitable at this point with Apple support in modern iPhones and iPads and Qualcomm, MediaTek, and Samsung adding ray tracing support in their devices as well in addition to desktop GPUs. While we acknowledge that the availability of this feature set is likely not likely to be present in public WebGPU apis for some time, I do believe that it is well established for the draft spec work to be started and investigations begun. I (and others here) are willing to some of the investigation, API and hardware comparison, and other required lead work if given guidance on the considerations needed. Please advise on the work needed to make some progress in this area. Thanks. |
|
Adding such a large feature to WebGPU will take considerable time, especially since to be useful ray-tracing needs bindless resources, which are just as complex a feature (if not more). That said we could start discussions if other members of the group are ok spending some of this group's time on it (it could be boxed to its own set of label, discussions etc so it doesn't derail WebGPU Milestone1/2 discussions). What you'd need to start is a comprehensive investigation of what the various APIs add for ray-tracing, what the similarities are and what the differences are, and ideally how developers expect to want to code against this extension. Then some iteration cycles between proposals and implementation experience, to finally get to something that's agreed upon. It will take a lot of time and effort. |
I've made a Ray-tracing implementation for the D3D12 (using DXR) and Vulkan (using VK_KHR_ray_tracing) backend of Dawn.
To demonstrate usage:
Link to specification here.
The text was updated successfully, but these errors were encountered: