Minutes 2022 04 13

GPU Web 2022-04-13

Chair: Corentin

Scribe: Brandon / Ken

Location: Google Meet

Tentative agenda

• Administrivia
  • Status of horizontal reviews.
  • Ok to start PING review (and landing Add GPUAdaperInfo to the spec #2660 before that)?
• CTS Update
• “Tacit Resolution” Queue
• Consider putting most of the writeTexture validation on the device timeline. #2680
• Cannot upload/download to UMA storage buffer without an unnecessary copy and unnecessary memory use #2388
• GPUBuffer.mapAsync is overly pessimistic #2646
• Concerns around error label and message #2633
• (stretch) Shader stage input-output subsetting rules #2038
• Agenda for next meeting

Attendance

• Apple
  • Myles C. Maxfield
• Google
  • Austin Eng
  • Brandon Jones
  • Corentin Wallez
  • Gregg Tavares
  • Ken Russell
  • Loko Kung
• Microsoft
  • Jesse Natalie
  • Rafael Cintron
• Mozilla
  • Jim Blandy
  • Kelsey Gilbert
• Mehmet Oguz Derin
• Michael Shannon

Administrivia

Status of horizontal reviews.

• MOD: i18n changes are ongoing.
• CW: It’s been over a year since we asked for TAG review. We also started security review. No news on that.

Ok to start PING review (and landing Add GPUAdaperInfo to the spec #2660 before that)?

• CW: Ok to start the PING review?
• MM: Would like a week to review.
• BJ: Sounds fine. We could start with just the design doc. Would be more valuable than the spec since it explains things better. But would be fine waiting a week if we can get the spec in.
• CW: Myles and Brandon to work together on this.

CTS Update

• CW: Kai is not here, but there’s a bunch of changes to make the CTS more correct/pass more tests. Fixed a bunch of Chrome errors. Myles also identified areas where the tests were incorrect. Still a ton of work to do. Help appreciated!
• MM: First step is making the tests pass, then making sure they’re exhaustive.

“Tacit Resolution” Queue

• 1. Change to createView() with no arg on 2D array textures #2684
  • Right now spec says creates view of single slice of texture
  • Change: if # array layers > 1, it's all of them - otherwise view of single slice
  • If # layers is programmatic - may need more changes
  • Alt proposal: could distinguish between 2D array and 2D texture object via GPU dimension. Would force the view, be more explicit.
  • Tacit res: change to the first thing I said.
• 2. #2729
  • Validation rule. Otherwise APIs have different behavior
  • If use texture format reinterpretation on attachment, & base format doesn't match base format of multisample target, APIs have differing behavior. Validate it out. Can't have one that's reinterpreted, and one that's not - base formats have to match.

Consider putting most of the writeTexture validation on the device timeline. #2680

• Differs in behavior from many other APIs in having validation on content timeline
  • In browsers, writeTexture wants to copy bytes to the GPU process. Needs estimation of # of bytes to copy.
• Would be non-normative note, browsers expected to do this estimation even though not written in spec.
• Formerly, Chrome would upload the entire Wasm heap to the GPU process every time. Bad idea. :)
• CW: moving validation from web process to GPU process.
• CW: will probably have to duplicate some computations of size from GPU process to web process.

Cannot upload/download to UMA storage buffer without an unnecessary copy and unnecessary memory use #2388

• MM: Can we postpone a week?

GPUBuffer.mapAsync is overly pessimistic #2646

• MM: imagine program where someone enqueues work that'll be fast using a buffer, then slow work to a different buffer, then maps the first buffer. Spec intentionally says the map operation will only be resolved once all GPU work's completed.
  • Pessimistic because buffer you're trying to map is idle, and ready to be mapped.
• MM: can we make this less pessimistic?
• MM: being less pessimistic - would make developers' programs go faster. More complicated, to know order of Promise resolutions. Need to know which work uses which resources.
• MM: historically, making stuff go faster's more important than having order of resolution be predictable. Solution would (probably) be consistent between browsers.
• MM: impl POV: for each set of work submitted, what resources does it act upon. Later, map request knows when we should resolve that Promise.
• BJ: is this something where the spec text could be loosened? Say, this may only resolve once all uses of this buffer have been resolved? Not strictly enforce the less pessimistic behavior? Or would this behavioral difference cause problems? Assume people might use buffer mapping as a pseudo-fence. E.g. expecting all work to be done on the queue.
• MM: mechanically, yes, spec could be looser & allow both behaviors. In general, WebKit team's experience is - whenever undefined order, it's bad. Prefer this be consistent between browsers.
• MM: fence operation you describe would only work if buffer you're mapping had any work referencing it. (?)
• KG: that's what we're talking about.
• MM: what I just said is false. Is it reasonable for someone to map a buffer just to see when work's done given that we have a function to run work on the queue when it's drained?
• CW: we use that in Dawn as fence substitute.
• CW: don't know it's possible to fully specify Promise resolution order. Problem: status of queue is in GPU process. Call you're making is in renderer process. Race condition. Please map buffer, last used at time 3, but: GPU process sending you message, I completed work up until 4. 3 should resolve before 4. But not possible to do because of message passing race condition.
• CW: we have shmem between 2 processes - maybe possible to do something - but avoiding adding locks would be ideal.
• CW: think not possible to specify this 100% I think.
• MM: impl strategy we're using - whenever someone calls map(), that'll involve a message to the GPU process. GPU process internally keeps track of all states. When map should be resolved, sends a message back.
• CW: is it sync?
• MM: no.
• CW: that's what I'm saying. At same time these 2 things happen, GPU process says, operations up until 4 are finished. Sends that message to the web process. Web process says, please map the buffer (once 3 is finished).
• MM: when web process says, mapAsync, GPU process will respond with when the Promise should be resolved.
• Assuming buffer map should resolve before onQueueSubmitted - how do you make this happen?
• MM: GPU process knows state of all resources. Knows if it's in flight or not. GPU process receives message, tell me when mapAsync should be resolved…can reply immediately, or add something to a list somewhere.
• KG: when Promise resolves, gets added to the Microtask queue. You'll get Promises queued up. Maybe things happen out of order.
• BJ: seems clear - on GPU side, can track where everything's been. Q is on renderer process side. Is your suggestion that we want to have strict ordering between when Promises resolve? Depending on architecture, might file mapAsync request, and also an onWorkCompleted request - might have a signal that can tell you, work's been done up to this point. Can resolve that one. But mapAsync may have to do a round trip. Could end up in a scenario - though buffer could have resolved before work signal was done, you'd get those two Promises out of order. Is this problematic in the scenario you're talking about?
• MM: I see. Could be a situation where content process says, onCompletedWorkDone, and also says mapAsync at the same time - and completed work Promise can be resolved immediately. But map call has to do round-trip to know.
• MM: think you're right, in such an architecture, ordering wouldn't be as deterministic.
• MM: approach we're taking - don't have the periodic message from GPU to content process. onCompletedWorkDone involves a round trip.
• CW: almost this. Except that onQueueSubmittedWorkDone by JS at some point, sent to GPU process. Then GPU process says queue's done - sends message. While that's in flight, content process sends mapAsync. The timelines / implications cross. On content process, would get onWorkDone, then mapAsync. No strict ordering because it's async communication.
• BJ: simplifies down to - we can probably write spec language saying mapAsync can resolve as soon as all work with that buffer's done. Not clear - order of Promise resolution on page side important? De-pessimizing mapAsync. But if ambiguity about Promise resolution - think it's architecture-dependent. Is it critical that those Promises resolve in the same order every time?
• MM: I don't know if I have good answer. Think ordering between multiple calls to mapAsync is more imp't than ordering between mapAsync and onSubmittedWorkDone.
• CW: happy to discuss this offline. Think I can find an example though even in WebKit's architecture where you can't 100% guarantee the ordering of Promises.
• MS: 1) Brandon's concern over using these as fences - we decided to do that in our engine. Reason we chose to do it that way - we only use fences to determine when buffer's ready in the first place. Best middle mapping. Concern, if people use our public API assuming it acts like a fence, e.g. for benchmarking. Didn't see too much issue otherwise.
• MS: 2) pretty sure Corentin's right about - without implementing some kind of sync, no way to get your maps in the same order you asked for them, or even the order they completed. Nice to rely on mappings coming back in the right order - not sure I'd have a use for onSubmittedWorkDone and mapAsync having an ordering. Easy to work around lack of ordering.
• MM: I think Corentin should write down an example of the claim. For use case of mapAsync as fence, think both options will work. Case that'll break - mapAsync on one buffer, assuming another buffer'll be idle (unrelated to first one).
• MS: not what we're doing. Being pessimistic about it.
• MM: which buffer do you call mapAsync on?
• MS: any one we want to read. Only use we had for fences. No use outside queries or readbacks. Use fences to know when buffers are ready.
• MM: is that different from onSubmittedWorkDone?
• CW: can I rephrase what MS is saying? Usually, in native-land, engines would have persistently mapped buffers, and fences to know when you can use certain sections. We don't give you access to the buffer unless that Promise is resolved anyway. Concept of fence -> concept of mapAsync. Also, when the browser lets you access the buffer. Fences are a native-engine-ism telling you the same thing.
• MM: as long as mapAsync is against buffer you're trying to read, both approaches should support that use case.
• MS: onSubmittedWorkDone can't be used because mapAsync can resolve after. Can't count on mapping being complete when onSubmittedWorkDone is complete
• CW: right now spec guarantees onSubmittedWorkDone and mapAsync Promises are resovled in order issued.
• MS: ah, we didn't plan on that.
• CW: talking about relaxing that here.
• CW: improving latency in these cases - can't define an ordering. Also - worried about add'l impl costs to track currently mapped buffers in flight. Not sure about developers' gains - can submit work & map right after. Can't really guarantee Promise resolution order, and devs can do this themselves - if we can guarantee, let browser do whatever they want - devs might map late on some browsers, might work well, or might not - add'l latency. Slight -1 for doing this. But also not the end of the world.
• MM: why don't you write your example, can take a week & come back.

Concerns around error label and message #2633

• MM: 3 pieces. 1) should we prefer saying in IDL "x or undefined" or "optional of x"?
• MM: 2) few places where we pass back human-readable strings through API. supposed to be a sentence. What should web page do with that?
• MM: 3) Marcos was confused about what .label field meant.
• CW: can we defer (1) to editors? WebIDL-ism.
• CW: interested in (2). Suggesting changes for this?
• MM: yes, changes would be - don't return String in the API. Take strings would have returned, print to console. Return boolean success/fail or similar. If we give readable string to JS program, it'll just show it to the user. User won't understand our error messages.
• BJ: one scenario we've seen with WebGL if we provide error messages- they can be captured & sent to server for debugging purposes. Devs looking for errors happening across range of hardware. Send back to server.
• BJ: agree not a fingerprinting concern. Error message will be the same for all Browser X instances.
• BJ: little reluctant to discount server-side debugging. Hard to get users to open console, copy/paste data. Otherwise, it's the only use case I can think of for allowing text capture. Don't think showing it to users will be useful.
• MOD: localization of string sort of overlaps with internationalization issue. Have language / direction fields sit next to these messages. Something to consider.
• MM: sort of the opposite direction. Rather than making error messages more friendly to show to users - think showing them is the wrong idea.
• MM: there are ways to gather telemetry in the browser. Private click measurement, etc. Don't need to send strings.
• CW: think we're discounting the shader editor use case.
• BJ: good point. Think shader errors are a different class.
• GT: think ShaderToy uses their own GLSL compiler. Info they show's more than comes back from WebGL.
• GT: localization - AFAIK JS syntax errors don't come back localized. "Identifier unknown" comes back in e.g. English, not Japanese.
• MOD: think there are some specs that do that. Intl team has pointed out certain specs that have these fields next to error messages, localized by browser directly.
• MM: even possible to do this? Catch JS error, get readable message out of it?
• BJ: JS errors do have a message field.
• MM: if we are reporting human-readable strings, shouldn't mandate they're written in English.
• CW: sounds fair. For WGSL errors for sure. Little cost to doing that in the spec.
• MM: 0 impl cost. Quality of impl concern.
• CW: we agree we need to keep WGSL errors exposed to JS. Non-controversial.
• GT: having an error localized to me - should be able to program keywords like "const" in other languages? If this changed to localized languages - library might say you have an undefined reference error, here's how to fix it. Or add constant referring to the string?
• CW: think I've seen Unix tools return localized errors.
• MM: are we saying we normatively say all error messages are in English? But not normatively dictate what they say?
• GT: I've had to parse strings to make devtools useful. Right now, not spec'd. Right now, matrix of 6 browsers' error messages - vs. 6 * 240 languages.
• CW: then please tell users to set browsers to English. Aside from language - codifying that …
• GT: don't think any other language localizes - Python, C++, Go, etc.
• MM: think we're agreeing that the spec doesn't dictate a language.
• KG: would rather that spec doesn't dictate anything.
• MOD: Intl team says error codes are preferred.
• MM: it's easy for Intl team to say - harder for us to agree on error codes. More reasonable - each browser can mint their error codes. Not compatible acros browsers. Browsers can add more fine-grained error codes.
• MOD: in that scenario, language/direction fields won't hurt it? Will satisfy intl team's request.
• KG: doesn't sound like what we want.
• MM: do we expect to show these to users? If no, don't think we need these fields.
• BJ: echoing what Myles said about error codes - lot of work, would look like what OpenGL looks like. 1 of 4 error codes, all unuseful.
• BJ: because of ShaderToy example - think there is a scenario where validation errors might be user-surfaced. Live editing like Codepen. JS errors in Codepen - those are surfaced inline. Can't really do that unless you're wrapping everything in ErrorScopes. But not unreasonable to think that Austin's live-editing code samples might want to surface validation errors. Might have error parser like Gregg suggested - but there's a scenario I see where they're useful.
• MOD: maybe keep issue open until we get Intl team's review?
• MM: do we have dialog with Intl team? Aware that these strings are not meant to be shown to users?
• CW: they are, sometimes.
• KG: they don't understand it, from their post. Not trying to be as prescriptive as we're concerned about. Not reaching each other in our dialog. Maybe online meeting with them would be better.
• MOD: By April 30 they'll come with complete feedback.
• KG: concerned complete feedback will be based on their post - based on incomplete understanding. Not even on the same page in this WG. Waiting until full response not useful.
• MM: agree. Think we need a dialog rather than a dump.
• CW: OK. Should we set up a new meeting with them?
• KG: if that's an option, think that'd be useful.
• KR: think we should leave the strings in there and give the info to applications.
• MM: think maybe device creation should tell browser which language they want the error messages in.
• MOD: reason why this happens - did this before - when I use an app and in middle of WebGPU process, you change the language - should report in the new language.
• CW: think it's fine to add a couple of fields next to things.
• KG: were bidi & language raised in the issue, or just in this meeting?
• MOD: I didn't bring it up as a comment on the issue, but I can.
• KG: if you're concerned about it, raise it, but if not, don't.
• MOD: just mentioning it in terms of relation with Intl team.

(stretch) Shader stage input-output subsetting rules #2038

Agenda for next meeting

• Carry over topics.