Allow format changes in copyTextureToTexture

[kainino0x]

Underlying APIs allow the source and destination textures to have different formats. We need to investigate the full matrix of allowed format pairs (and aspects) for copies.

• It would be convenient to be able to copyTextureToTexture between the srgb and non-srgb versions of a format (e.g. rgba8unorm <-> rgba8unorm-srgb). Regardless of whether we get view reinterpretation in Proposal: Add a usage bit to allow creating a view with different format when creating a texture #168, this is helpful. (Copies operate on textures, not texture views.)
• Contradictions in the validation rules of copyTextureToTexture() with depth stencil formats #1464 highlights spec issues with depth/stencil format copies that would get resolved by this.

[kainino0x]

Starting a spreadsheet here https://docs.google.com/spreadsheets/d/1mRunoiNt_YGNPpwJYzJ3TdhLeQ6yO0DU_ezRmgNAMx8/edit?usp=sharing

[kainino0x]

I didn't complete the spreadsheet because the rules are fairly succinct.

Vulkan:
vkCmdCopyImage requires format compatibility

• Uncompressed color formats: Between any two with the same block size (even rg11b10, rgb9e5, etc.)
• Compressed color formats: Between two that differ ONLY in srgb-ness and signed-ness.
• Depth/stencil: Format must match exactly. Aspect may be D, S, or D+S (I think).
• EDIT (see below): Between uncompressed and compressed color formats that are "size-compatible" despite apparent contradiction with other rules

Metal:
Format must match. A texture view (#744) is required to change format.

• Without a format-reinterpretation flag set at createTexture time, textures may differ ONLY in srgb-ness.
  • Unclear what happens when the swizzle differs. Sounds like the channels get swizzled during the copy, i.e. R and B don't get swapped. Swizzle doesn't matter because it doesn't let you change the format itself from RGBA8 to BGRA8.
• With the flag set (Proposal: Add a usage bit to allow creating a view with different format when creating a texture #168):
  • Uncompressed color formats: Between any two with the same block size.
  • Compressed color formats: Between two that differ ONLY in srgb-ness.
  • Depth/stencil: Format must match exactly. Must copy all aspects.

D3D12:
Formats must be compatible (excel spreadsheet).

• This seems to be the least restrictive, but I didn't dig deeply because of the limitations set by Metal. It still allows changes in srgb-ness (uncompressed or compressed).
• Not sure it's possible to copy only some aspects.

---

In the default case, Metal is clearly the most restrictive: we can copy between srgb<->non-srgb of the same format.

EDIT: Removed stuff about swizzling, realized that doesn't apply to copies.

[kainino0x]

Another thing to consider: The underlying APIs don't allow e.g. copying depth32float-stencil8's depth aspect into depth32float. In native, you can just copy with an intermediate buffer. But in WebGPU that's impossible: copies from buffer into depth32float aren't allowed because we can't validate the depth range (unless a future unrestricted-depth feature is enabled).

We could make such copies possible by implementing them with an internal intermediate buffer.

EDIT: filed as #2328.

[kainino0x]

PR #2329 opened based on my investigation.

[fintelia]

Vulkan also allows copying between compressed and uncompressed images:

vkCmdCopyImage allows copying between size-compatible compressed and uncompressed internal formats. Formats are size-compatible if the texel block size of the uncompressed format is equal to the texel block size of the compressed format. Such a copy does not perform on-the-fly compression or decompression. When copying from an uncompressed format to a compressed format, each texel of uncompressed data of the source image is copied as a raw value to the corresponding compressed texel block of the destination image. When copying from a compressed format to an uncompressed format, each compressed texel block of the source image is copied as a raw value to the corresponding texel of uncompressed data in the destination image. Thus, for example, it is legal to copy between a 128-bit uncompressed format and a compressed format which has a 128-bit sized compressed texel block representing 4×4 texels (using 8 bits per texel), or between a 64-bit uncompressed format and a compressed format which has a 64-bit sized compressed texel block representing 4×4 texels (using 4 bits per texel).

[kainino0x]

Thanks, I missed that! It seems to contradict the sentence just 2 paragraphs prior that I actually read:

The formats of srcImage and dstImage must be compatible. Formats are compatible if they share the same class, as shown in the Compatible Formats table.

So I didn't read any further.

[kvark]

@kainino0x thank you for investigating this!
It sounds like, with Metal being the common denominator, we have to resolve #168 first, and then piggy-back on it.

[kainino0x]

Reopening for larger classes of format reinterpretation in post-v1