Proposal for Handling Native/Webgpu Extensions and Expanding Limits

[cwfitzgerald]

Introduction

As #690 implements the first wgpu native extension, it's important that there is a plan in place on how to implement additional extensions.

For the purpose of this discussion:

• webgpu extension refers to an extension that is standardized in webgpu.
• native extension refers to an extension to wgpu that is not part of the upstream webgpu standard. It may optionally be supported by other native webgpu implementations, such as dawn.
• unsafe extension refers to an extension that could make running untrusted code unsafe, or cause UB. These would inherently be native extensions.

A couple considerations to keep in mind, especially regarding native extensions:

• It should be hard for gecko/servo to accidentally use a native or unsafe extension. Their goal is to implement WebGPU, so it should be easy for them to disable all native extensions. Additionally, enabling an unsafe extension could result in a major security hole, which no one wants.
• It should be very clear to wgpu-rs users that asking for a native extension is not portable to the web.
• Enabling an unsafe extension should demand unsafe code to at turn it on, though the interface provided by the extension should also be unsafe if possible.

Proposal

With those considerations in mind, I propose the following way of implementing extensions.

NonExhaustive Member

There will be a struct in wgpu-types which is defined as

#[doc(hidden)]
pub struct NonExhaustive(#[doc(hidden)] pub ());

There will be a note on the struct saying that manual construction of this type is not protected under our semver rules, and if you construct it, your code may break at any point. This type will not be reexported from wgpu-rs, so most users cannot use it.

Types declared in this proposal as NonExhaustive shall have a public member of this type to prevent direct construction without a partial construction + default.

struct SomeStruct {
   // members
   pub _nonexhaustive: NonExhaustive,
}

This is because #[non_exhaustive] doesn't allow users to construct structs manually, even partially. Additionally, due to the desugaring of partial construction (rust-lang/rust#63538) private fields are not allowed to be constructed either. No matter the method, it still leaves us between a rock and a hard place, as this type needs to be fully constructable from wgpu-core, and rust doesn't consider wgpu-core any different than any other user of wgpu-types.

Extensions Struct

The extensions struct will be a single bitfield that contains all the extensions that are in webgpu or supported by wgpu. Native extensions will always report false on the web, and unsupported webgpu extensions will always return false.

bitflags::bitflags! {
    pub struct Extensions: u64 {
        const ANISOTROPIC_FILTERING = 0x0000_0001_0000_0000;
        const WEBGPU_EXTENSIONS = 0x0000_0000_FFFF_FFFF;
        const NATIVE_EXTENSIONS = 0xFFFF_FFFF_0000_0000;
    }
}

Unsafe extensions

Unsafe extensions be normal members of the Extensions bitflags. When you create a device, there will be an extra argument to device creation which must be provided to enable the use of unsafe extensions. The type would look like this:

#[derive(Default)]
struct UnsafeExtensions { allow_unsafe: bool }
impl UnsafeExtensions {
    pub fn disable() -> Self { Self { allow_unsafe: false } }
    pub unsafe fn enable() -> Self { Self { allow_unsafe: true } }
}

If any unsafe extensions are requested, allow_unsafe must be true.

We follow the nomicon's definition of rust's safety. You must not be able to invoke UB from safe rust. There must be unsafe in order to invoke it. Additionally, the UB should not occur too far from the use of unsafe, if at all possible

Limits Struct

The limits struct is NonExhaustive.

Extensions and limits can interact in a couple of ways:

Extension adds limit

If an extension adds a limit, it will be added to the Limits struct as is. If the limit has a sane value for "not enabled" the limit will be that value when the extension is disabled. For example, anisotropic_filtering could have a associated limit of max_sampler_anisotropy. If the extension is disabled, it will be 1, representing no anisotropy. If the limit has no sane default value, it will be an Option<T> and the limit will be None if the extension is disabled.

Extension turns webgpu restriction into limit

Webgpu is (necessarily) more restrictive than the underlying apis as it needs to run on all apis. In wgpu, we can expose the underlying api/hardware's restriction directly. This situation is most easily explained with an example.

In webgpu, there is a requirement that all offsets into buffers are 256 byte aligned. gfx-hal exposes the actual alignment requirement of the underlying api/hardware. A native extension could be designed: relaxed_uniform_buffer_offset_alignment which adds a new limit uniform_buffer_offset_alignment. If the extension is requested, and granted, the limit value will be the value of the underlying hardware. If the extension is not requested or not supported (web), the limit will be the value of the current webgpu restriction.

This streamlines both implementation and usage. These limits will often be hardcoded into both wgpu and user code as constants. In both cases, that constant will be replaced with reading the value of the limit. Because the limit defaults to the current webgpu restriction and gecko/servo will not enable native extensions, this will ensure webgpu validation remains correct. Additionally, user code will be optimal and portable across both native and web.

Extension removes limit

I am including this for completeness, but I honestly can't think of a situation where this would actually happen.

Descriptor Structs

All descriptors that are possibly extendable are NonExhaustive.

As discussed in #689, any additional data needed for an extension will be directly added to the wgpu-types struct, likely as an Option<T>. Setting the value to anything other than None requires the extension be enabled.

In wgpu-native, each extension gets its own extension struct, chained together with pnext pointers. This is to be consistent with how webgpu extensions will have their own structs.

Member functions

Extensions might add member functions to various components. These will always exist, but have an assert validating that the extension is enabled on call.

Thoughts?

This is my idea for how extensions should be implemented. Details are of course, open for modification to best serve users/implementation.

I'd love to hear feedback from both maintainers and users :)

Edit: Updated with the results of a discussion on matrix with @kvark

Edit: Updated 2020-06-03

[kvark]

@zakarumych raised a concern about unsafe extensions. Opting into them with unsafe only at initialization makes it difficult to reason about safety down the road, i.e. anything can be unsafe, implicitly. This is bending the concept of unsafe in Rust a bit too much, and is not sound. For example, if there is a middleware layer, and the user app gets access to, say, command buffers, it has no means to discover that some operations on those are unsafe.

I think the reasoning makes sense. We should remove the UnsafeFeatures mechanics and instead let them add unsafe methods to wgpu-core.

[cwfitzgerald]

Fwiw, my understanding of the plan was to do both UnsafeFeatures + unsafe interface for unsafe extensions. This provides a check to make sure that unsafe things can't just be freely called without some change in extension init. If you'd still rather not do the first part, that's still perfectly fine. I agree that unsafe needs to be as local as possible both for soundness and for reasonable debugging.

[cwfitzgerald]

I think that we have implemented enough extensions at this point that we can call this implemented :)