lookAt() Not working when used in onModelLoaded callback

[obolland]

As title suggest, lookAt function doesn't work as expected when used inside the onModelLoaded callback. Does it need to be used in a different way?
Would be nice to get this working

[brianzinn]

I suspect it is not due to loader and callback, but the forward direction. Have a look here:

https://github.com/brianzinn/react-babylonjs/blob/master/storybook/stories/babylonjs/GUI/with2DGUI.stories.js#L135

notice the extra parameters and the yaw/pitch/roll corrections in radians:

https://doc.babylonjs.com/typedoc/classes/babylon.abstractmesh#lookat

[obolland]

I've experimented with the yaw/pitch/roll to no avail. What works for one co-ordinate does not work for another. Also weirdly, if I set the model's position via the position props e.g...

          <Model
            sceneFilename="something.gltf"
            rootUrl='path'
            position={new Vector3(10, 0, 10)}
            onModelLoaded={onModelLoaded}
          />

and then use the lookAt() function inside the onModelLoaded callback, the model orientates itself differently than if I set the model's position inside of the onModelLoaded callback instead of using the API for setting the position, despite the positions being identical. Something odd is happening...

Also worth noting that if I load the model instead inside of the onSceneMount callback using SceneLoader.ImportMesh() and then use the lookAt() function, it works exactly as expected using the same model and position

EDIT I now have it working, but only if the position is set from within the onModelLoaded callback. If the position is set via props, looAt() doesn't work for some reason 🤷‍♂️

[brianzinn]

I would need to see your callback, but the Model uses the rootModel to assign all properties. Model loaders other than glTF do not necessary have a "root" mesh, so a root mesh is created and position/transform are applied on that mesh. I don't know if that could be the underlying cause. There is not a lot going on in the Model itself except calling a hook that calls SceneLoader (this ties into React.Suspense for the fallback):
https://github.com/brianzinn/react-babylonjs/blob/master/src/customComponents/Model.tsx

Have a look in your callback here:

const onModelLoaded = (loadedModel) => console.log(loadedModel.rootMesh);

I know there can be differences between setParent() and .parent = ... - setParent maintains position in world space. Main thing to note is that setting position via a prop may be different than setting position on the model itself.

Does that explain the difference to you?

[obolland]

Yes, thank you @brianzinn 👍😊
Loving react-babylonjs by the way

[brianzinn]

Maybe for 'gltf' I can just use the __root__ mesh. I think it's just an empty mesh with -1 z scaling for babylon right-handed.

[obolland]

So what you said made sense until I looked closer and now I'm at a loss again...
The model in question is a gltf model. If I apply a position via the props or apply a position via the callback, the exact same position is applied to the rootMesh, as expected. However, the lookAt() function applies a different rotation value to the rootMesh depending on whether the position was set via props or the callback. I don't know how lookAt() works behind the scenes, but I'm totally at a loss as to why/how this is happening!
My code looks more or less like this...

           <Model
            sceneFilename="myModel.gltf"
            rootUrl="./assets/models/"
            position={new Vector3(20, 0, 20)}
            onModelLoaded={(model, lookAtPoint) => onModelLoaded(model, lookAtPoint)}
          />

and the onModelLoaded callback...

      export const onModelLoaded = (model, lookAtPoint) => {
      const myModel = model.rootMesh
      // arrow.position = new Vector3(20, 0, 20)
      myModel.lookAt(lookAtPoint, Math.PI / 2);
 }

The rotation on the root mesh remains untouched in each approach.
I'm not sure if this is a bug or I'm just missing something? But any help would be much appreciated!

[brianzinn]

I’ll make a sandbox. Are you able to share your mesh?

[brianzinn]

hi @obolland I found the issue - turns out that I didn't need your model to find the problem. I believe you are calling .lookAt(...) and then the reconciler is setting the position.

const LookAtModel = ({ lookAtPosition, position, id }) => {
  let baseUrl = 'https://raw.githubusercontent.com/KhronosGroup/glTF-Sample-Models/master/2.0/';

  const modelRef = useRef(null);

  const onModelLoaded = (model) => {
    console.log('model loaded', model, lookAtPosition);
    modelRef.current = model.rootMesh;
    // I'm setting the position here.
    modelRef.current.position = position;
    modelRef.current.lookAt(lookAtPosition.clone())
  }

  useEffect(() => {
    if (modelRef.current) {
      modelRef.current.lookAt(lookAtPosition);
    }
  }, [lookAtPosition])

  return (
    <Suspense fallback={<box name='fallback' position={position} />}>
      <Model rootUrl={`${baseUrl}Avocado/glTF/`} sceneFilename={`Avocado.gltf?id=${id}`} scaleToDimension={1} position={position} onModelLoaded={onModelLoaded} />
    </Suspense>
  )
}

export const LookAtStory = () => {
  const [lookAtPosition, setLookAtPosition] = useState(Vector3.Zero());

  return (
    <>
      <div style={{ flex: 1, display: 'flex' }}>
        <button onClick={() => setLookAtPosition(new Vector3(0, lookAtPosition.y === -2 ? 2 : -2, 0))}>Look Up, Down</button>
      </div>
      <div style={{ flex: 1, display: 'flex' }}>
        <Engine antialias adaptToDeviceRatio canvasId='babylonJS'>
          <Scene>
            <arcRotateCamera name='camera1' alpha={Math.PI / 2} beta={Math.PI / 2} radius={9.0} target={Vector3.Zero()} minZ={0.001} />
            <box position={lookAtPosition} size={1} />
            <hemisphericLight name='light1' intensity={0.7} direction={Vector3.Up()} />
            <LookAtModel lookAtPosition={lookAtPosition} position={new Vector3(3, 0, 3)} id='1' />
            <LookAtModel lookAtPosition={lookAtPosition} position={new Vector3(-3, 0, -3)} id='2' />
          </Scene>
        </Engine>
      </div>
    </>
  )
}

Note that I am calling .position = ... before .lookAt(...) otherwise what is happening is that you are having the model look at a position and then the reconciler is moving the model (and the rotation doesn't change after the position).

I think the solution will be that I apply all of the props supplied (ie: position) and then call the onModelLoaded or apply the props before a new callback onAfterModelLoaded (or something), if I wanted to maintain backwards compatibility. I think it's ok to not maintain backwards compatibility. Can you confirm that fixes your issue and if so also share any thoughts on the callbacks?

[obolland]

Thanks @brianzinn for looking at this. You're absolutely right. And yes, I can confirm that forcing the reconciler to apply the position changes to the model after props have been applied fixes the issue.
I had wrongly assumed that the props would be applied before the callback ran, so I would personally say that applying all props before calling onModelLoaded should be the expected behaviour. What's your opinion?
Thanks again,
Olly

[brianzinn]

Agree with you that would be expected behaviour - I'll apply props before callback and drop a release tonight. Cheers.

[brianzinn]

Should be fixed - the props are also applied after the callback, so that might be interesting. I nearly went with an opt-out mechanism, but will see how it goes. Can you confirm it's working as expected @obolland ? cheers.

[obolland]

Thanks @brianzinn 👍
So far so good! Working totally as expected. Great stuff 😊