Sprites and imperative/declarative mix

[namklabs]

Hello,

I apologize in advance for the lengthy issue but I am very confused about this.

I need to use Sprites and I wanted to use react-babylonjs, but I can't figure out how to do it or if it's even possible. I understand from this thread that sprites might not be fully implemented yet (although "needs improvement" left me wondering), and that opened up a question in my mind about determining exactly what is implemented thus far.

I guess my big question is how to determine which Babylon JS classes I can implement in a JSX React-like fashion and which ones I must do imperatively. I haven't found a comprehensive list of components that react-babylonjs offers as wrappers for Babylon JS elements.

Here's an example of why I'm so confused. This example imports { Vector3, Color3, Color4, Animation, ExponentialEase, EasingFunction, Texture } from '@babylonjs/core' and imports only Engine and Scene from react-babylonjs.

However, this example also implements the following Babylon JS classes in JSX/React component style without actually importing them:

<arcRotateCamera/>
<arcRotateCamera/>
<box/>
<standardMaterial/>
<plane/>
<rectangle/>
<stackPanel/>
<babylon-button/>
<advancedDynamicTexture/>
<vrExperienceHelper/>

Where are these coming from if they aren't imported? I see that a custom Components folder is being exported but the files in there don't appear as I would expect:

Another confusing example: Importing StandardMaterial to use it imperatively but then <standardMaterial/> is used decralatively in the React tree. What makes StandardMaterial different from <standardMaterial/>, and where does <standardMaterial/> come from?

I want to know so I can look at all the other available React components.

I tried to use <sprite/> and <spriteManager/> and it was a big fail.

Compounding my confusion is the fact that these components start with a lowercase letter. React docs say "User-Defined Components Must Be Capitalized".

Thank you for any clarification you can provide!

Edit: linked to note about sprites needing improvement: #6 (comment)

[brianzinn]

hi @namklabs -- I found what you are experiencing confusing at first as well. The first iteration of this project did not use the react-reconciler and, in fact, all the components were like <Scene ../> and <Engine ../>, which are declared like regular React components.

The main difference is that what we are loading inside the <Scene .../> that are not Capitilized are not React components - they are host components. Just like a <div /> or <span /> for the ReactDOM renderer, we are now loading BabylonJS components as host components - they are not attached like DOM elements and have their own lifecycle (ie: we may call dispose()) when they are removed from the scene.

There is a comprehensive list of what components are implemented on the project main readme:
https://github.com/brianzinn/react-babylonjs#babylonjscore-api-support

They way they are added is that this project examines the typings for the babylonjs projects (core/gui) and creates everything the reconciler needs to pick up elements like <standardMaterial /> and connect it to the correct babylonjs object - the reconciler also knows how to automatically attach a material to a mesh and a texture to a material - using the metadata and information from the code generation (also how to instantiate objects from props by knowing their constructor args, for example) - and to flow prop changes directly to underlying BabylonJS object in your scene graph.

I can explain further anything further that isn't clear on that part.

Now your Sprites are not implemented, but I am happy to add them. Can you come up with a declarative syntax that is intuitive and will meet your needs? If so, let's try to get it added :) It would need to follow the same naming guidelines as the actual BabylonJS objects.

Cheers.

[namklabs]

Hello @brianzinn ! That explanation was a tremendous help. Thank you so much.

I guess when I saw the list of implemented components in the main readme I was hung up on the lowercase thing and the lack of importing, so it all makes sense now. Would it help if I forked and added some explanatory detail to the readme?

As for Sprites, I can see why they would be tricky to implement via a declarative syntax since they need a SpriteManager, but how does this look?

import sonicTheHedgehog from './sonic.png';
...skipping ahead...
<Engine>
    <Scene>
        <spriteManager name="sonicManager" imgUrl={ sonicTheHedgehog } capacity={ 1000 } cellsize={ { width: 280, height: 410 } } />
        <sprite name="sonic-a" manager="sonicManager"/>
        <sprite name="sonic-b" manager="sonicManager" position={new Vector3(1,2,3)} />
        <mesh name="sonicOverHere" position={new Vector3(5,10,20)}>
            <sprite name="sonic-c" manager="sonicManager"/>
        </mesh>
    </Scene>
</Engine>

I don't like how the SpriteManager just hangs around in there but I couldn't think of anything better. Let me know what you think!

While we are on the subject, could you give me an example of how I could add a Sprite to my declarative scene with the Babylon API? There's gotta be a way to do it, right?

Something like this (I know this won't work):

<Engine>
    <Scene>
        <sphere name="filledWithSprites">
            { this.generateSprites( this.state.numberOfSprites ) }
        </sphere>
    </Scene>
</Engine>

[brianzinn]

import React { useRef, useEffect } from 'react';

const MyComponent = (props) => {
  const sphereRef= useRef(null);

  useEffect(() => {
   const sphere = sphereRef.current;
   const mesh = sphere.hostInstance;
   // add sprite stuff here imperatively
  }, [sphereRef]);

  return (
    <sphere ref={sphereRef} />
  )
}

That should be close, I'm just typing from memory - something like that.

If there is a Playground then maybe that is a good inspiration? I usually start with a BabylonJS playground and then work backwards. The other thing is that the Sprite Manager and Sprites don't need to be direct descendants in the tree. I did a lot of tricks in the gui 3d manager and with shadows and physics, so things don't need to be directly connected - sometimes that makes it more intuitive. I think what you have there with the manager name may be enough :)

[brianzinn]

About the docs - I already want to remove all of the examples from the docs and turn it into more of a tutorial about how it works. So, really any PRs on the readme in that direction would be accepted. What I want the readme to do is walk through how everything works and building something in the end.

[namklabs]

@brianzinn thanks so much for all the help!

[makar-al]

I know that this issue is closed, but I'm still interested, are there any plans to add sprites to react-babylonjs?

[brianzinn]

hi @makar-al - no plans, but I could add that - do you have a declarative proposal? Basically we could start with what the imperative code is and then I can add the declarative parts. I suspect you would want to create a manager and then maybe sprites declared inside that space would be assigned the manager they are contained in (or manager could be a property)? Maybe make a new issue as a feature request for tracking?

[makar-al]

@brianzinn Thanks for a quick response. I think manager should be some kind of a hook or a wrapper around provider and sprite can be just a simple element that takes a manager returned by a hook as a prop. Ok, I'll make a new issue.