feat: 140 on demand rendering

[alvarosabu]

This PR introduces the concept of conditional rendering. It closes #140

The renderMode prop has been added to the TresCanvas component with 3 different options:

• always the default one (current solution)
• on-demand will automatically invalidate the frame on:
  • Instances prop changes
  • Context state change (resize, clearColor etc)
  • remove nodes (v-if)
• manual: the invalidate method needs to be called by the user to trigger a render.

TresCanvas now emits a render event:

<TresCanvas
    render-mode="on-demand"
    clear-color="#82DBC5"
    @render="onRender"
  >

I'm still finishing the docs, you can start taking a look to it.

[alvarosabu]

TresCanvas now emits a render event:

<TresCanvas render-mode="on-demand" clear-color="#82DBC5" @render="onRender" >

Maybe should we include an beforeRender And afterRender ?

[andretchen0]

Hey! I'm having a look.

Maybe I just need more context, but on a first read through, I'm wondering:

For the useTres() API, would it make sense to merge advance and invalidate? Maybe under invalidate? And then accept invalidate(n?) in both on-demand and manual modes? (Currently invalidate only works in on-demand mode and console.warns otherwise.)

(My mental representation of this functionality is "mark frame as dirty", so "invalidate" makes more intuitive sense to me than "advance".)

Merging under invalidate has the slight advantage that it doesn't require a different API call if the render-mode is changed. So something like this ...

watch(() => props.myProp, invalidate)

... can just chug right along. Whereas currently, I think we'd need to switch over the render mode, e.g.:

watch(() => props.myProp, () => {
    if (renderMode === 'manual') {
        advance()
    } else if (renderMode === 'on-demand') {
        invalidate()
    }
})

[alvarosabu]

Hi @andretchen0 thanks for the feedback. Honestly I had the same dilemma as you, I added the advance method just because is the same approach used by Threlte and R3F

I think is because when you are on-demand the primary role of invalidate is to signal that the scene needs to be re-rendered. It essentially "flags" the scene as invalid or dirty, indicating that something has changed and a re-render is necessary.

When invalidate is called, it doesn't directly cause the scene to render immediately. Instead, it sets up the conditions for the scene to be rendered in the next iteration of the render loop.

The advance function is used to manually progress the rendering process by one step. It's a more direct and immediate way to trigger a render and update cycle.

That being said, maybe makes more sense to change advance to directly render instead of marking a new render

 function advance() {
    if (rendererOptions.renderMode === 'manual') {
      renderer.value.render(scene.value,camera.value)
    }
    else {
      logWarning('`advance` can only be used when `renderMode` is set to `manual`')
    }
  }

Regarding switching between render mode on the fly, do you see it as a common use-case?
wdyt?

[andretchen0]

Honestly I had the same dilemma as you, I added the advance method just because is the same approach used by Threlte and R3F

Gotcha. No problem. If other libraries are using a separate method, then maybe there's a good reason.

Regarding switching between render mode on the fly, do you see it as a common use-case?
wdyt?

I don't see it as a common use case, but people are surprising! ;)

[alvarosabu]

Hi @andretchen0 and @Tinoooo, I'm currently facing some performance issues on the devtools (creating the scene graph and reactivity) because I'm sending an event to the devtools every time it renders.

Should we set "on-demand" rendering as default? wdyt?

[thomasaull]

Maybe a fourth mode would make sense, where the render stops, when the current tab/window is not active anymore. This would prevent the scene rendering in the background, eating up performance and draining the users battery. The hasFocus and/or visiblityState APIs could be used for this.

It's also worth mentioning, that vueuse has abstractions for this. If you don't want to add a dependency, maybe it could still provide some pointers etc…:

• https://vueuse.org/core/useWindowFocus/#usewindowfocus
• https://vueuse.org/core/useDocumentVisibility/#usedocumentvisibility

[andretchen0]

Hi @andretchen0 and @Tinoooo, I'm currently facing some performance issues on the devtools (creating the scene graph and reactivity) because I'm sending an event to the devtools every time it renders.

Should we set "on-demand" rendering as default? wdyt?

I'd prefer "always" as the default, if possible. "always" matches my starting assumption for how rendering works: always throw away the last frame and render a new frame.

Assuming my understanding of "on-demand" is correct, I think I'd be confused if I e.g., wrote a shader and then found it wasn't updating consistently because it wasn't hooked to a prop.

[alvarosabu]

You are right, also, we would need to refactor practically the ecosystem to be on-demand first, like OrbitControls and such.

Let's keep always.

[alvarosabu]

@thomasaull That is a brilliant idea, I would probably add it to one of the current states instead of a new one. ☝️

[alvarosabu]

Maybe a fourth mode would make sense, where the render stops, when the current tab/window is not active anymore. This would prevent the scene rendering in the background, eating up performance and draining the users battery. The hasFocus and/or visiblityState APIs could be used for this.

It's also worth mentioning, that vueuse has abstractions for this. If you don't want to add a dependency, maybe it could still provide some pointers etc…:

• https://vueuse.org/core/useWindowFocus/#usewindowfocus
• https://vueuse.org/core/useDocumentVisibility/#usedocumentvisibility

@andretchen0 @thomasaull @Tinoooo what do you think? Should we create a new rendering mode like on-visible or add this functionality to on-demand rendering?

[thomasaull]

@andretchen0 @thomasaull @Tinoooo what do you think? Should we create a new rendering mode like on-visible or add this functionality to on-demand rendering?

Hmm, I could imagine use cases for all modes that you want to pause rendering in the background. If it were up to me I'd probably add another prop like renderInBackground or pauseRenderInBackground to TresCanvas

[andretchen0]

Maybe a fourth mode would make sense, where the render stops, when the current tab/window is not active anymore. This would prevent the scene rendering in the background, eating up performance and draining the users battery. The hasFocus and/or visiblityState APIs could be used for this.
It's also worth mentioning, that vueuse has abstractions for this. If you don't want to add a dependency, maybe it could still provide some pointers etc…:

• https://vueuse.org/core/useWindowFocus/#usewindowfocus
• https://vueuse.org/core/useDocumentVisibility/#usedocumentvisibility

@andretchen0 @thomasaull @Tinoooo what do you think? Should we create a new rendering mode like on-visible or add this functionality to on-demand rendering?

Is this not the way TresCanvas works currently? If I'm understanding properly, TresCanvas uses vueuse's useRafFn, which in turn uses the browser's requestAnimationFrame. requestAnimationFrame leaves it up to the user's browser, but generally speaking, calls are paused when e.g., the tab is hidden.

(Fwiw, I mostly work on Cientos and I'm not familiar with a lot of the Tres internals, so maybe I'm not looking/tinkering in the right spot.)

[alvarosabu]

@andretchen0 good point, you are right, I just check how many times it renders when I change tab and it stopped.

[thomasaull]

@andretchen0 @alvarosabu In my tests with babylonjs (other engine but should be similar) this worked well, when the tab was in the background yes. But using multiple windows where the window running the engine is visible but not focused left it running at 60 FPS. Given many 3d websites/applications are very resource intensive I'd still consider adding the option to pause rendering when the tab/window is not focused.

[andretchen0]

@thomasaull @alvarosabu

Given many 3d websites/applications are very resource intensive I'd still consider adding the option to pause rendering when the tab/window is not focused.

Sounds handy to me.

Maybe getting to big for this PR, but ...:

• To piggyback on @thomasaull 's suggestion, some projects like this one include a "focus helper" feature that e.g., darkens the screen and displays a message to let the user know the window doesn't have focus and won't respond to keyboard/mouse/touch events. I've found that to be quite useful for users and even for myself during development.
• For render modes, it'd be lovely if we had a mode where canvases stopped updating/rendering if not visible on screen, e.g., scrolled off-screen. For Cientos docs, we've started including multiple canvases on a single page in order to show off different capabilities/props. They update/render even if they can't be seen.

[alvarosabu]

For render modes, it'd be lovely if we had a mode where canvases stopped updating/rendering if not visible on screen, e.g., scrolled off-screen. For Cientos docs, we've started including multiple canvases on a single page in order to show off different capabilities/props. They update/render even if they can't be seen.

That's why I want to consider it. But I agree with you, maybe we should create a different ticket for it. @thomasaull or @andretchen0 would you be so kind as to create a feature request?

@andretchen0 can we merge this on v4?

[andretchen0]

For render modes, it'd be lovely if we had a mode where canvases stopped updating/rendering if not visible on screen, e.g., scrolled off-screen. For Cientos docs, we've started including multiple canvases on a single page in order to show off different capabilities/props. They update/render even if they can't be seen.

That's why I want to consider it. But I agree with you, maybe we should create a different ticket for it. @thomasaull or @andretchen0 would you be so kind as to create a feature request?

Will do.

@andretchen0 can we merge this on v4?

I'll have a look at the code later today.

[andretchen0]

@alvarosabu

Nice job! Looks to me like everything's working. I wasn't able to break the functionality.

The comments are either typos or some code quality stuff that you can accept or ignore – I won't block the PR. It's been lingering long enough; I don't want to stand in the way.

[andretchen0]

For future readability/understandability, can we make this more descriptive? It's linked to renderMode, so maybe renderModeState?

[alvarosabu]

@andretchen0 if the name renderModeState is meant to add clarity that the property isn't just a static setting but potentially a reactive state that could change over time, then we should rename most of the properties inside the TresContext type to state, like controlsState and rendererState etc.

Imo being inside of the TresContext which is already a state implies its nature without the need to add extra verbosity.

[andretchen0]

Hey @alvarosabu

I'd be fine with dropping State off the end, as most variables are some sort of state. (But then the name renderMode will collide with the other renderMode.)

The current name – internalState – doesn't say anything about what the object is used for. Since it's about renderMode-stuff, I thought that'd make it more obvious to future readers.

But no problem leaving it as is.

[alvarosabu]

@andretchen0 True, internal doesn't give enough context. 🤔 Let's continue this exercise, then, what if we rename the InternalState to RenderState and add everything related to it inside?

export interface RenderState {
  mode: Ref<'always' | 'on-demand' | 'manual'>
  priority: Ref<number>
  frames: Ref<number>
  maxFrames: number
  canBeInvalidated: ComputedRef<boolean>
  invalidate: () => void
  advance: () => void
}

I think it would be cleaner.

[andretchen0]

Sounds good!

[andretchen0]

Like the type, can we make this field name more descriptive? Maybe renderModeState?

[alvarosabu]

Hey, buddy @andretchen0 I refactored internal for render, what do you think?

[andretchen0]

Hey, buddy @andretchen0 I refactored internal for render, what do you think?

Hey! I think that's clearer than internal. You?

[alvarosabu]

Way better