How to set raycasters in React 360?

[zzc-tongji]

I try to use the same way as react-vr, but fail.

// client.js

// This file contains the boilerplate to execute your React app.
// If you want to modify your application's content, start in "index.js"

import {ReactInstance} from 'react-360-web';

function init(bundle, parent, options = {}) {
  const r360 = new ReactInstance(bundle, parent, {
    // Add custom options here
    fullScreen: true,
    raycasters: [{
      getType: () => 'simple',
      getRayOrigin: () => [0, 0, 0],
      getRayDirection: () => [0, 0, -1],
      drawsCursor: () => true
    }],
    cursorVisibility: 'visible',
    ...options,
  });

  // Render your app content to the default cylinder surface
  r360.renderToSurface(
    r360.createRoot('Hello360', { /* initial props */ }),
    r360.getDefaultSurface()
  );

  // Load the initial environment
  r360.compositor.setBackground(r360.getAssetURL('360_world.jpg'));
}

window.React360 = {init};

Is there any way to set raycasters in React 360? It is really important for users of mobile browser and Google Cardboard to interact with React 360 websites ("click" something).

[andrewimm]

Yes, raycasters are still customizable – if anything, they are far more configurable with React 360 now.

Raycasters now must implement this interface: https://github.com/facebook/react-360/blob/master/React360/js/Controls/Raycasters/Types.js
They behave mostly the same, the biggest difference is that raycasters now fill an existing array, rather than returning a new one each frame. This is a significant performance improvement. Raycasters can also now return their location and direction in terms of camera coordinates (for something like a gaze cursor) or world coordinates (for something like Oculus Touch). I'd recommend looking at the files in https://github.com/facebook/react-360/tree/master/React360/js/Controls/Raycasters for examples.

Raycasters are no longer initialized at start, since you may want to change them after React has been initialized. r360.controls (seen here: https://github.com/facebook/react-360/blob/master/React360/js/Controls/Controls.js) exposes three functions you may find helpful:

• addRaycaster() registers a new raycaster at the end of the list.
• clearRaycasters() removes all raycasters
• getRaycasters() returns a copy of the list of raycasters

Between the three of these, you should be able to achieve any desired effect.

[zzc-tongji]

I have read all articles and codes you mention above, and then tried a lot. However, I still fail to create a raycaster for users of mobile browser and Google Cardboard.

Could you provide an example?

[andrewimm]

I have provided three full examples in this repository, available at https://github.com/facebook/react-360/tree/master/React360/js/Controls/Raycasters

Your code needs to implement all of the methods. I will try to explain more about each one here.

• drawsCursor returns a boolean determining whether you should see the cursor on a button or surface. This is useful for controller and gaze cursors, but not for mouse or touchscreen.
• fillDirection(direction) determines the 3D vector for the direction of the cursor. If the Raycaster is not active, return false. If the raycaster is active, put the x, y, and z elements of the direction vector into the array passed as the first argument, and return true. React 360 assumes that a raycaster that returns false has not modified the array, and a raycaster that returns true has modified it.
• fillOrigin(origin) is identical to the above. If setting the origin, put the x, y, and z elements in the array passed as the first argument, and return true.
• getMaxLength()returns a numeric length for the raycaster. Typically this is just Infinity
• getType() returns a unique string identifier for the raycaster. This will be important when we enable multiple cursors.
• hasAbsoluteCoordinates() returns a boolean that determines whether this raycaster exists in camera space or absolute space. A raycaster in camera space moves with the camera, like a gaze cursor. A raycaster in absolute space does not, like Oculus Touch, which is independent of the camera.

If all of these are implemented, your raycaster should be usable. The easiest way to test is to clearRaycasters() to eliminate any conflicts, and addRaycaster() with an instance of your custom implementation.

[mathdroid]

I have created a new example for react-vr-gaze-button that implements simple-raycaster using the new method, and it works well. You can see it here: https://github.com/mathdroid/react-360-gaze-button/blob/master/example/client.js

By the way, I think the fillDirection and fillOrigin methods can be nicer if we can just return a new array instead of mutating the parameter. What do you think @andrewimm

[andrewimm]

@mathdroid the fill behavior is a very intentional architectural choice made throughout the new Runtime. If you return a new array every single frame, you very frequently encounter garbage collection, which can lead to "hiccups" in rendering. We are working towards building our own renderer, and entirely eliminating memory allocation each frame. If we can reach that, React 360 will only allocate new memory when objects are created, which means that the average experience will see significantly improved performance, especially on low-end devices.

[ibenarobeno]

@mathdroid thanks for an example. How to get the cursor visibility working with your example?

[etabrizi]

Anyway we can have a raycaster to work for 3D models using location? Or can the gaze event work for 3D models and not just 2D surfaces?

[danielstorey]

When I add a simple raycaster (for the purpose of adding gaze buttons) nothing happens unless I call r360.controls.clearRaycasters(); first. But then I lose my mouse raycaster and nothing's clickable.

I suppose I could add a mouse one back in manually but is there an easier way to add the simple raycaster without having to remove the others.

I am working only on desktop for this part of the development process.