Consider supporting DOM elements inside <Canvas>

[sxxov]

Thank you for the project!

Explanation

Currently, the <slot /> inside <Canvas> puts all children into the DOM <canvas>'s children. Whilst this, makes sense to some degree, it really is kind of pointless, as all DOM elements just become fallbacks for browsers who don't support <canvas> (IE8??).

I'm working on a project that mixes DOM elements alongside threlte components, all under a global canvas. I've needed to resort to plenty o' hacks, including manual scroll restoration, stubbing files containing @threlte/* imports for SSR, & remounting the entire tree into a sibling portal on hydration.

Having support for such a mixed use-case would be great, as it means a jankless SSR-ed DOM, & threlte would just be loaded during hydration. More precisely, making the 'threlte' & other 'threlte-*' contexts available, in a stubbed way during SSR through an isometric useThrelte(), or introducing a new sub-package like @threlte/core/iso that makes client-side-only hooks return nullable values during SSR to shift the stubbing responsibility to the consumer. Then, <T> components could become passthrough noops & complete the circle.

Example

Source

<!-- App.svelte -->

<script>
  import { Canvas, T, useThrelte } from '@threlte/core';
</script>

<Canvas>
  <!-- during SSR, maybe this either, returns an isometric interface that doesn't render anything, or returns null/throw? -->
  {useThrelte(), ''}

  <!-- this would probably just be a pass-through noop, rendering its children -->
  <T.Group />
  <T.Group>hello</T.Group>

  <!-- you get this both during SSR & CSR, as a sibling to `<canvas>`, kinda like `@threlte/extras/HTML` -->
  <h1>world</h1>
</Canvas>

Expected Result

<canvas></canvas>
hello
<h1>world</h1>

Actual Result

<canvas>
  hello
  <h1>world</h1>
</canvas>

[stefnotch]

Another use-case for this would be tweakpane libraries.

Actual

Whenever I want to tweak the value of a component, I have to register the tweaker outside of the scene. (App.svelte)
And then it needs to be passed as a prop into the scene. And sometimes it needs to be passed into further children. That's a lot of glue code to set up a one-off development tool.

e.g. https://threlte.xyz/docs/examples/geometry/random-placement#basic-random has examples where the tweakpane usage could be simplified

Expected

I'd like it if I could simply register a tweakpane anywhere inside a <Canvas>. For example, taking the following setup

• App.svelte: Entry point
• Scene.svelte: Imports Random.svelte
• Random.svelte: Generates a scene procedurally

If I want to tweak the const commonRatio = 0.5 parameter for procedural generation, then this should be possible

1. Change it to let commonRatio = 0.5, and hook it up
2. Add a tweakpane slider to Random.svelte
3. Try out different values interactively
4. Now hardcode the new value const commonRatio = 0.42
5. And remove the tweakpane slider from Random.svelte

This would involve adding and removing around 3 lines of code, instead of messing with a store, or hooking up props.

[DefinitelyMaybe]

There is some progress regarding the usage of svelte-tweakpane-ui in our documentation, which improves the ease of setting up interactive variables. There are also promising solutions like triplex and the already integrated theatre.js for interactive variables/code.

However, I am not fully understanding this issue.

Currently, only <a/>, <button/>, and <input/> elements are permitted within a canvas, as stated in the MDN documentation.

I am curious about the purpose of mixing DOM elements with threlte components. Is it related to tweakpane functionality or is there another reason behind it?

[sxxov]

@DefinitelyMaybe

My particular use-case is having inline 3D rendered graphics, that use a shared canvas throughout the lifecycle of the app.

Mayhap the live version of the aforementioned project might be clearer.

[DefinitelyMaybe]

My particular use-case is having inline 3D rendered graphics

excellent.

that use a shared canvas throughout the lifecycle of the app.

Oh, do you mean like the <View /> component in pmndrs/drei ?

[sxxov]

@DefinitelyMaybe

that use a shared canvas throughout the lifecycle of the app.

Oh, do you mean like the <View /> component in pmndrs/drei ?

Some API like that would be ideal! Right now i’m rawdogging coordinate calculations all over the app. I’m a bit concerned that nested views may hinder SSR, but once again, react folks get it right.

[franknoirot]

+1 for a component like <View />. I'd be happy to try and implement it with a little onboarding if it's not something too fundamentally off the aims of threlte! Or would you say threlte is more aimed at standalone THREEjs apps written in the manner of Svelte/SvelteKit apps, not many little ones embedded in a larger Svelte app?

Although my use case doesn't look nearly as cool, I am facing a similar issue to @sxxov with my little app https://text-to-cad.zoo.dev (repo here), where I'm trying to be kind to users by not loading all their CAD models at once in the list view. I'm only showing a couple canvasses at once thanks to an IntersectionObserver, but I'm still getting hit with Too many active WebGL contexts. Oldest context will be lost. and other WebGL context weirdness.

I think a global <Canvas /> context and many <View /> components is just what I need. I naively tried wrapping the <slot /> in my +layout.svelte in a Canvas but that of course removed all my UI.

[DefinitelyMaybe]

That'd be lovely @franknoirot !

@grischaerbe already help me with converting the component a while ago but it's taken me some time to get around to merging it in. Happy for you to take it the last mile if you're up for it.

There are some small things regarding the target dom's CSS props absolute/fixed that should be part of our docs but will discuss within the PR.

Here's a link to the component if you're ready to get started

[franknoirot]

Excellent, that's so great that you've already done so much of the component conversion, thanks! So is FunBit aiming to be the Svelte equivalent to drei? I'm gonna solve my work problem along a different avenue real quick and then take a crack at it.

[DefinitelyMaybe]

Little bit off topic but not at all. Funbit is just making quaternus assets easier to muck around with