We need something like requestFinalFrame() to run logic after all *Observer callbacks

[trusktr]

The problem is clearly described in

• Add BoundingClientRectObserver #9104
• UPDATE: new issue specifically describing the problem and possible solutions: [resize-observer] why we need ResizeObserver.takeRecords() or ResizeObserver.hasRecords() (the canvas flickering problem) w3c/csswg-drafts#9717

but I'll paste it here too (click to expand):

But there's a problem with adding new *Observer patterns in general:

We have no way to derive state from multiple combinations of them for use in the browser paint cycle.

The more *Observer APIs we add, the more need we have for some way to run logic after all paint-cycle observers have ran (by paint-cycle observers, I mean those that run in sync with animation frames and browser paint).

Here's the problem, which add yet another frame-based *Observer will exacerbate:

At present, if we rely on an animation frame, a ResizeObserver, a MutationObserver, and an IntersectionObserver, in order to react to changes of all of those, and to derive all the state we need for rendering something to a WebGL canvas, this means we will have to render to the canvas a total of 4 times in one frame, reducing our framerate to 25% (ouch!!!!!). In each callback, starting with the animation frame callback, we have to update state and then render.

We must render in every type of callback, because we don't know, for any given render frame, which callbacks will run.

If we render to canvas only in an animation frame callback (which is essentially the pattern that everyone (and I mean practically everyone) is writing today), we'll run into unintuitive issues like this one where on every resize the canvas will flicker (typically a white color on websites with default CSS background).

Here's a demo. Set SHOW_FLICKER_PROBLEM to true to see the issue here:

https://codepen.io/trusktr/pen/EzBKYM

Here's what the problem looks like:

flicker.mov

Then set SOLVE_FLICKER_PROBLEM to true to enable the solution, and the problem will be gone:

no-flicker.mov

Note the comments, which will point out where the canvas is double rendered during resize, cutting the framerate of a resize in half (ouch!!). Resizing browser windows is typically sluggish already, and this will make things worse.

This problem happens because if we resize a canvas in a ResizeObserver callback after we have already rendered in an animation frame callback (animation frame callbacks always fire before ResizeObserver callbacks) and do not render again in the resize observer callback, then the canvas pixels will be cleared. The pixel clearing is a natural consequence of resizing a canvas and has nothing to do with animation frames or resize observers, however the way these APIs work, we do not have a reliable way to handle certain state changes after multiple callback groups (anim frames, observer callbacks).

The only way to get around all issues with aimation frames and *Observer APIs with simple code is to resort to polling for all state in a single animation frame callback, but with the added performance cost of unnecessarily reading state every frame (although in some cases this can be faster than double rendering).

To avoid polling, the solution would be very complicated, requiring use of MutationObserver to detect every possible change in the DOM that could possibly a client rect to change (there are many many possibly ways to change a client rect including <style> content changes, style attribute changes including of parents and children, inside and outside of shadow roots, etc, it gets really really complicated to do it robustly without missing any cases).

TLDR

Yes, new APIs like ClientRectObserver (I think this name is fine without "Bounding" in it) would be nice, but we also need a way to run final logic after all frame-based *Observer callbacks so that adding yet more *Observer APIs doesn't make the above issue worse.

For example, maybe something like this:

requestAnimationFrame(function loop() {
  // ... update things like position/rotation of objects, etc ...
  requestAnimationFrame(loop)
})
new ResizeObserver(() => {...}).observer(el)
new ClientRectObserver(() => {...}).observer(el)
new MutationObserver(() => {...}).observer(el) // microtask, irrelevant to render loop

requestFinalFrame(function finalLoop() {
  // This runs after everything, even frame-based *Observer APIs added in the future.
  // ... calculate final state, finally render to canvas a SINGLE time ...
  requestFinalFrame(finalLoop)
})

Every render cycle of the browser, callbacks would be called in roughly this order:

• animation frame callbacks (do not render to canvas here)
• *Observer callbacks in some particular order, but we should not rely on the order, just collect needed state.
• final callbacks (derive final state, render to canvas, etc)

Solution

Something like requestFinalFrame()would allow running a frame after animation frame callbacks and *Observer callbacks (including callbacks of any frame-based *Observer APIs added in the future, and potentially any other type of frame-based callbacks added in the future that currently don't exist), so that final state and rendering can be performed in a consistent manner without duplication of final state derivation or rendering logic (see the problem and codepen example in the other issue).

// in the next browser render frame, the given callback will run *after* animation frame
// callbacks and `*Observer` callbacks such as for ResizeObserver, allowing us to perform
// final tasks such as rendering final derived state to a canvas:
requestFinalFrame(() => {
  // ... render to canvas a *single* time *after* all state, including canvas size, is derived ...
})

[annevk]

I don't understand why you can't schedule updates and make them during requestAnimationFrame.

Also, this would benefit a lot from reading through https://whatwg.org/faq#adding-new-features and applying it. Thanks!

[trusktr]

I don't understand why you can't schedule updates and make them during requestAnimationFrame.

@annevk Wish it was that easy! Some observer APIs that run callbacks between rAF and browser paint (f.e. ResizeObserver, and potentially new observer APIs like BoundingClientRectObserver and ComputedStyleObserver) introduce the problem described in the following issue (examples included):

• [resize-observer] why we need ResizeObserver.takeRecords() or ResizeObserver.hasRecords() (the canvas flickering problem) w3c/csswg-drafts#9717

With requestFinalFrame (or requestPaintFrame or some similar named API), the convention for web developers would become:

• animate data with requestAnimationFrame
• handle further data updates in observer callbacks without worrying about drawing yet (f.e. ResizeObserver callbacks)
• finally use requestFinalFrame (or requestPaintFrame or whatever name we come up with) to draw/render a single time, f.e. resizing a canvas and finally sending WebGL draw commands.

[greggman]

Hadn't seen this thread, posted some solutions here

https://stackoverflow.com/questions/77842752/how-do-i-use-resizeobserver-with-requestanimationframe-correctly

[Kaiido]

One problem I can see with yet another callback in the event-loop, it that the changes made in that callback would get missed by all these observers. Then we'll want a FinalObserver of some sort to tackle these, etc.

I feel the best would be to have a way to limit the possible actions in this hook to compositing related actions only, so maybe a Worklet like PaintWorklet would actually be the best place for such updates, though the limited API of this latter is certainly an issue.