Minutes 2022 08 17

GPU Web 2022-08-17

Chair: KG

Scribe: KR

Location: Google Meet

Tentative agenda

• Administrivia
• CTS Update
• “Tacit Resolution” Queue
• Specify that texture_external sampling functions clamp coords to [0, 1] #2766
• GPUBuffers shouldn't have to accumulate a list of mapAsync promises to reject #3271
• Colorspace conversions for GPUExternalTexture are arbitrarily complex #3293
• Frame advancement of canvases that aren't visible #3295
• Buffer state validation in unmap() is unnecessary #3251
• maxColorAttachments not sufficient? #2965
• Agenda for next meeting

Attendance

• Apple
  • Myles C. Maxfield
• Google
  • Brandon Jones
  • Gregg Tavares
  • Kai Ninomiya
  • Ken Russell
• Microsoft
  • Jesse Natalie
  • Rafael Cintron
• Mozilla
  • Kelsey Gilbert
• Mehmet Oguz Derin

Administrivia

• Kelsey OOO 2 weeks from now, Corentin and Kelsey trading chairing responsibilities
• Congratulations to new Editors!
• KN: lot of discussion about finalizing the spec in last week’s CG meeting. Myles, please review meeting minutes.

CTS Update

• KN: not a lot. Ryan's working on shader builtin tests, moving to vector-related builtins. API side, Gyuyoung from Igalia's working on validation tests. Going through the spec and finding validation we didn't have tests for, keeps finding things, this is good.
• Shrek & I fixed some canvas readback tests after alpha mode change.
• Had a bug in Chrome's WebGPU -> WebGL tests where they were vertically flipped incorrectly. Fixed the tests and implementation; the y-flip is gone. Now when you read back a canvas into a WebGL texture the orientation's the same no matter whether it was WebGPU-rendered.

“Tacit Resolution” Queue

Limit for the maximum buffer size · Issue #1371

• Limit for max buffer size
• KN: finished discussing, didn't record resolution. Myles hadn't finished looking into it, but later posted a comment saying we should add this limit. Unanimous agreement? Please comment.
• KG: what should the limit be? 640 MB. :)
• KN: we can bring it back when we have a number.
• MM: there is a number in Metal, part of the PDF table that's published (actually not - see below). Reason we have this limit - no concept of - attaching a buffer to a shader, you attach a part of it. In Metal we attach the entire buffer. In Vulkan there's a limit of the accessible part of a buffer from the shader.
• KG: per-buffer max?
• MM: yes.
• KN: we can get back to this one.
• KN: will we do 256 MB?
• KG: feels low, but sure.
• MM: appears to not be in the PDF document. It's an attribute on the Metal device.
• KG: makes it harder?
• KN: MoltenVk might publish it, also Stack Overflow.
• KG: sounds like general consensus to add a number for this. Let's find what it should be.

Consider making ShaderModule compilation errors special · Issue #2119

• KN: think where we ended up: remove ordering of promise resolution from PopErrorScope, and don't need to make ShaderModule compilation errors special. Can just defer resolution of the Promise.
• MM: Ken had some feedback.
• KN: think Ken's feedback was that as long as it's designed to not block forward progress, it's OK.
• BJ: linked in last comment on this issue - know we've socialized it elsewhere before. Goes over the reasoning. One of biggest reasonings - will be easy for us to decide, we want these well ordered. But going other way will be difficult.

https://github.com/gpuweb/gpuweb/pull/2301

• Proposal for pipeline statistics.
• BJ: sitting for a while, got approval from Dzmitry, me, Kelsey, Corentin. Just want to say, either we should land this, or toss it - not have it sit open forever. Probably waiting for Myles to say yes.
• MM: I think it's fine, I'll approve it right now.
• BJ: great, thank you.
• MM: I like the fact that this is stated not part of the spec and not final.
• KN: I wrote it, i like it too. :)

https://github.com/gpuweb/gpuweb/pull/3014

• buffer state as attribute
• KN: this is the PR addressing the issue. 3 states: mapped, pending, unmapped. Don't have to worry about multithreading yet. Can figure that out later. Same as labels.
• KG: thumbs up. Forward progress. We're actively investigating on our side - if we don't like it, we'll change it again.
• MM: fine with us.

https://github.com/gpuweb/gpuweb/pull/3123

• "out-of-memory" -> "internal" error type
• BJ: discussed last week. Concern: still nice to know when OOM is OOM. Latest version adds an enum to the interface, says reason for internal error. Either "OOM" or "undefined". Will be undefined for most things, like shader compilation failing with internal error.
• MM: surprisign development. Last I heard, thought we wanted to unify these because people who wanted to know the distinction would wrap distinct API calls with ErrorScopes.
• KG: disagree philosophically with that, should call OOM OOM.
• KN: we have just two ErrorScope types now.
• BJ: Myles, that formulation was the Editors' last resolution; presented to the CG, got feedback, we'd agreed we could add an enum for this.
• MM: why is an enum better than a third error type?
• KN: we should punt this then.

https://github.com/gpuweb/gpuweb/issues/3297

https://github.com/gpuweb/gpuweb/pull/3311

https://github.com/gpuweb/gpuweb/pull/3318

https://github.com/gpuweb/gpuweb/issues/3324

Specify that texture_external sampling functions clamp coords to [0, 1] #2766

• MM: proposal I made - rather than representing addressing modes in the sampler, name them in the functions in the shading language or an enum
• KN: I don't agree there's a problem with having a texture sampling function that ignores part of the sampler. It's really clear. Someone coming to WebGPU for the first time - it says it does repeat, and it does.
• KN: What's the chance they'll use a sampler with an external texture? Pretty clear to users that samplers apply to both regular and external textures. This problem only applies to external textures. If they use a sampler with a regular texture, they'll understand the difference.
• MM: now two sources of information, the name of the function, and the sampler? Which one wins?
• KN: They have to type something in the shader that makes it clear something different is happening here. Not an implicit change any more - the point of this rename.
• KG: not sure how to make forward progress here. Would like to, by constraining what we offer in V1 to something we have consensus for, while allowing expansion in the future. My main constraint - don't want to offer REPEAT for this. Would be happy to not support anything but CLAMP. Willing to compromise with CLAMP + REPEAT_MIRRORED, but not REPEAT.
• MM: going back to goal hierarchy - primary goal to let authors specify something they support (?). Second is to specify REPEAT. If Kelsey doesn't want REPEAT on the web platform, …
• KG: people who care about impl need to come up with a proposal matching our constraints. We can't just keep handing each other proposals and saying no. Need a counter-proposal.
• KN: problem I have here - don't wantt o make a compromise actively detrimental to the API for the sake of what I consider theoretical purity. Goal #1 - theoretical purity. Clear naming, clear to developers there's a difference - consider the concern that we're dropping part of the sampler for this particular sampling call to not affect developers. Don't want us to take a concrete tradeoff for that.
• KG: in exploring the solution space - why are we trying to have a textureSample function for external textures, when we want texelFetch?
• KN: that'd be a concrete tradeoff. When you have YUV textures, you can do filtering on the Y + UV textures rather than the RGB result. Should be cheaper.
• KG: feels you'd want a different sampler to tell you linear at what point. Don't think we define where this filtering happens.
• KN: we want it to be undefined. Per our media team, difference in practice is imperceptible. Want to leave both options open. Forcing user to filter after sampling is annoying for developers and not good for performance. Don't think it's a good compromise.
• KG: what if we had a texture sample level that didn't have a sampler object? For texture external, didn't have a sampler object? If you didn't want linear sampling, you should have done texel fetch?
• MM: think that's worth considering.
• KN: there'll still need to be a sampler object somewhere. Equivalent to the option of where we turn external texture into combined texture+sampler. Tradeoffs - extra sampler in every external texture.
• MM: all such samplers will be identical. Could be a static inside the shading language.
• KN: would love to do that, but can't do that in other languages.
• KG: if we picked only one of these - we'll just give you linear, clamped sampling unless you use texelFetch - if you want NEAREST, use texelFetch. I can make a PR for this.
• KN: interesting idea. In practice, where would we put that sampler? Once per bind group? Impl has to find someplace for it itself, doesn't count toward limits?
• MM: thing you mentioned before about constructing samplers at runtime isn't a part of any shading language. In Metal they have to be constexpr.
• KN: could always have 3 or 4 texture functions, but point taken.
• MM: in Metal we already have a bunch of shadow data for buffer inputs - it's standard practice for Metal.
• KN: problem is, it's not just data. Can't just append it to a uniform buffer we already have. Need another slot.
• KG: let me think about it, get back with a proposal.
• KN: could do one per bind group. My opinion about this being theoretical purity still stands.
• RC: one reason why the external thing needs to be known/special - in CHromium impl, it comes with add'l metadata. Like - YUV/RGB conversion necessary, rotation, clipping necessary. WebGPU runtime needs to know these things. When you sample, it does all these things behind your back. This sort of abstraction is what makes this external thing better and faster.
• KN: we want to take advantage of the sampling hardware. Not so simple.

GPUBuffers shouldn't have to accumulate a list of mapAsync promises to reject #3271

• Jim OOO
• KN: Right now, all map requests are queued up
• KN: The proposal is to replace the map with a single item. On client side, if map request outstanding, we’ll just not let you make another one. I think this is fine. I think this just slightly changes the timings of rejections, and this seems fine to me.
• MM: Right, we’re already tracking this in the web process, as we agreed earlier today.
• MM: I think this is a good change.
• KN: In the future, if we allow multiple ranges, we wouldn’t need to enforce order on the mapping.
• MM: This is kinda backwards from how we decided that compilations aren’t special. There we said we would enforce ordering even if things happen out of order. Here, we’re saying that order just doesn’t matter. Just an observation.
• KN: I don’t think I see the parallel.
• KN: Actually, we said we wouldn’t enforce ordering.
• (comment from MM.)
• KN: Return the first error in the scope yes, but no ordering of promise resolution. Outer scope can resolve before inner scope.
• Resolved: Take this

Colorspace conversions for GPUExternalTexture are arbitrarily complex #3293

• MM: in discussions with internal team. Can we defer to next week?

Frame advancement of canvases that aren't visible #3295

• BJ: Kai and I spent a long while digging through HTML specs. Had report from someone who was using Chrome's impl that if the canvas wasn't visible they kept getting the same texture back. It's what the spec said. Could choose not to paint.
• BJ: frame boundaries - not in HTML spec. After lots of proposals, came to - after you request getCurrentTexture(), will schedule microtask as part of the same operation to expire the texture. Will disappear at the end of the callback.
• MM: isn't that soon? There are many microtasks that happen in a single frame.
• BJ: there are. But - the concept of "frame" is not very well defined. In HTML spec there are 2 branches you could consider a frame. Depends on whether something's visible. Don't want - I'm going along, expecting this behavior - and someone hid the tab, slid it offscreen, etc. If HTML spec chooses not to paint, not a lot of reliable places to hook these. Taking away access to texture handle at end of task seems reliable enough, not detrimental to anyone.
• MM: what does WebGL do?
• KG: end of rAF?
• KN: actually, WebGL spec says - if preserveDrawingBuffer:false, all bets are off as soon as the task ends.
• MM: task, not microtask.
• BJ: we could schedule this as a task. Kai had reasons for preferring microtask.
• KG: what's the problem with the behavior we have?
• BJ: it's not predictable or consistent. Doesn't apply uniformly between windows and workers.
• KN: in essence - when you update the canvas's rendering, we take away access to the frame. Might not happen - might have a hitch - timing's unreliable. Event loop's one task, then maybe composites, etc. If we try to define it based on compositing occurring, there's no guarantee about the tasks that happened before or afterward. Depends on the browser compositing.
• MM: there's an "udpate the rendering".
• BJ: there is. we asked about that. processing model - process a task, process that task's microtasks, then page has a rendering opportunity. If page chooses to and can present its content (vsync-aligned), goes through the rest of the compositing tests. More fuzzily defined than you might think. If you didn't get around to rendering - you wouldn't get a new texture. In CTS and one of our partners - they had a canvas not attached to the DOM, so never qualified for the rendering opportunity, and got the same texture back forever. Not the behavior we want.
• KG: actually I think that's fine.
• KN: has to be a way to advance the next frame.
• KR: you have to be able to capture the canvas into a MediaStream, for example.
• KN: now that you mention it, canvas readbacks aren't subject to this.
• BJ: if you aren't compositing and return the same texture over and over again, then you attach it - might have N frames of input rendered over itself. First frame will potentially have a flash of accumulated content.
• KG: are we trying to do better then WebGL does?
• BJ: we have to be better then WebGL, which is undefined.
• KN: have to do better than WebGL, there's a state change visible to the app.
• KR: there are multiple ways of potentially displaying canvases - regular canvas, OffscreenCanvas transfer (main thread or worker), MediaCapture -> video element, …
• BJ: (had a question)
• KG: problem I see with this - like "all rendering must happen on the main thread". Feels like the same flavor. Do all your rendering in the same callback loop / task. Worries me - because you touched this, you have to finish with it, have to do all your rendering now.
• BJ: don't want to take away Myles' chance to speak This is true today - when you get the texture, you're on the clock, and the clock is unpredictable. It's when the compositing's able to happen. Here we're proposing a way - we can modify it - for the clock to be predictable. If we want people to be able to grab a texture and do as much as they want with it - then we want an explicit swap and I don't think we want that.
• KG: not sure we don't want that
• MM: difference between onscreen etc - not unique to 3D rendering / WebGL / WebGPU. E.g. minimize window, rAF doesn't fire. We have more info than rAF does. But - another example, custom paint has callbacks that fire at a certain point. Observable, and there's a policy about when / whether they fire. Think we should consult wiht CSS custom paint authors.
• KN: what we're trying to build on here is WebGL's prior art.
• MM: don't want to judge whether things should never move on if the window's minimized. But if we do wnat frames to march on when elements' not attached, then should be at the microtask boundary.
• KN: q is, can they rely on those being in the same task.
• MM: they can.
• KN: I think it's a rare case.
• KR: we consulted with Domenic, an HTML spec editor on this - IndexedDB hooks in to the HTML rendering loop for example in an undesirable way. Domenic advised to queue a microtask for this purpose. The behavior will be consistent, specifiable, and testable. It is an error if applications expect to be able to rAF and setTimeout and access the same WebGPU backing texture in both.
• (more discussion, no conclusion)

Buffer state validation in unmap() is unnecessary #3251

maxColorAttachments not sufficient? #2965

Agenda for next meeting

• Specify that texture_external sampling functions clamp coords to [0, 1] #2766
• Colorspace conversions for GPUExternalTexture are arbitrarily complex #3293
• Frame advancement of canvases that aren't visible #3295
• Buffer state validation in unmap() is unnecessary #3251
• maxColorAttachments not sufficient? #2965