Skip to content

pmndrs/react-xr

master
Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?
Code

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
src
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

@react-three/xr

Version Downloads Discord Shield

React components and hooks for creating VR/AR applications with @react-three/fiber

npm install @react-three/xr

Examples

These demos are real, you can click them! They contain the full code, too.

Getting started

The following adds a button to start your session and controllers inside an XR manager to prepare your scene for WebXR rendering and interaction.

import { VRButton, ARButton, XR, Controllers, Hands } from '@react-three/xr'
import { Canvas } from '@react-three/fiber'

function App() {
  return (
    <>
      <VRButton />
      <Canvas>
        <XR>
          <Controllers />
          <Hands />
          <mesh>
            <boxGeometry />
            <meshBasicMaterial color="blue" />
          </mesh>
        </XR>
      </Canvas>
    </>
  )
}

XRButton

<XRButton /> is an HTML <button /> that can be used to init and display info about your WebXR session. This is aliased by ARButton and VRButton with sensible session defaults.

<XRButton
  /* The type of `XRSession` to create */
  mode={'AR' | 'VR' | 'inline'}
  /**
   * `XRSession` configuration options
   * @see https://immersive-web.github.io/webxr/#feature-dependencies
   */
  sessionInit={{ optionalFeatures: ['local-floor', 'bounded-floor', 'hand-tracking', 'layers'] }}
  /** Whether this button should only enter an `XRSession`. Default is `false` */
  enterOnly={false}
  /** Whether this button should only exit an `XRSession`. Default is `false` */
  exitOnly={false}
  /** This callback gets fired if XR initialization fails. */
  onError={(error) => ...}
>
  {/* Can accept regular DOM children and has an optional callback with the XR button status (unsupported, exited, entered) */}
  {(status) => `WebXR ${status}`}
</XRButton>

XR

<XR /> is a WebXR manager that configures your scene for XR rendering and interaction. This lives within a R3F <Canvas />.

<Canvas>
  <XR
    /**
     * Enables foveated rendering. Default is `0`
     * 0 = no foveation, full resolution
     * 1 = maximum foveation, the edges render at lower resolution
     */
    foveation={0}
    /** Type of WebXR reference space to use. Default is `local-floor` */
    referenceSpace="local-floor"
    /** Called as an XRSession is requested */
    onSessionStart={(event: XREvent<XRManagerEvent>) => ...}
    /** Called after an XRSession is terminated */
    onSessionEnd={(event: XREvent<XRManagerEvent>) => ...}
    /** Called when an XRSession is hidden or unfocused. */
    onVisibilityChange={(event: XREvent<XRSessionEvent>) => ...}
    /** Called when available inputsources change */
    onInputSourcesChange={(event: XREvent<XRSessionEvent>) => ...}
  >
    {/* All your regular react-three-fiber elements go here */}
  </XR>
</Canvas>

useXR

This hook gives you access to the current XRState configured by <XR />.

const {
  // An array of connected `XRController`
  controllers,
  // Whether the XR device is presenting in an XR session
  isPresenting,
  // Whether hand tracking inputs are active
  isHandTracking,
  // A THREE.Group representing the XR viewer or player
  player,
  // The active `XRSession`
  session,
  // `XRSession` foveation. This can be configured as `foveation` on <XR>. Default is `0`
  foveation,
  // `XRSession` reference-space type. This can be configured as `referenceSpace` on <XR>. Default is `local-floor`
  referenceSpace
} = useXR()

To subscribe to a specific key, useXR accepts a Zustand selector:

const player = useXR((state) => state.player)

Controllers

Controllers can be added with <Controllers /> for motion-controllers and/or <Hands /> for hand-tracking. These will activate whenever their respective input mode is enabled on-device and provide live models for a left and right XRController.

<Controllers
  /** Optional material props to pass to controllers' ray indicators */
  rayMaterial={{ color: 'blue' }}
  /** Whether to hide controllers' rays on blur. Default is `false` */
  hideRaysOnBlur={false}
/>
<Hands
  // Optional custom models per hand. Default is the Oculus hand model
  modelLeft="/model-left.glb"
  modelRight="/model-right.glb"
/>

useController

useController references an XRController by handedness, exposing position and orientation info.

const leftController = useController('left')
const rightController = useController('right')
const gazeController = useController('none')

XRController

XRController is an Object3D that represents an XRInputSource with the following properties:

index: number
controller: THREE.XRTargetRaySpace
grip: THREE.XRGripSpace
hand: THREE.XRHandSpace
inputSource: XRInputSource

Interactions

To interact with objects using controllers you can use <Interactive /> component or useInteraction hook. They allow adding controller event handlers to your objects.

Interactive

<Interactive /> wraps your objects and accepts XR controller event handlers as props. Supports select, hover, blur and squeeze events (see XR inputsources).

<Interactive
  /* Called when hovered by a controller */
  onHover={(event: XRInteractionEvent) => ...}
  /* Called when unhovered by a controller */
  onBlur={(event: XRInteractionEvent) => ...}
  /* Called on button press when selected by a controller */
  onSelectStart={(event: XRInteractionEvent) => ...}
  /* Called on button release when selected by a controller */
  onSelectEnd={(event: XRInteractionEvent) => ...}
  /* Called on button release when another interactive is selected by a controller */
  onSelectMissed={(event: XRInteractionEvent) => ...}
  /* Called when selected by a controller */
  onSelect={(event: XRInteractionEvent) => ...}
  /* Called on button press when squeezed by a controller */
  onSqueezeStart={(event: XRInteractionEvent) => ...}
  /* Called on button release when squeezed by a controller */
  onSqueezeEnd={(event: XRInteractionEvent) => ...}
  /* Called on button release when another interactive is squeezed by a controller */
  onSqueezeMissed={(event: XRInteractionEvent) => ...}
  /* Called when squeezed by a controller */
  onSqueeze={(event: XRInteractionEvent) => ...}
  /* Called when a controller moves over the object, equivalent to pointermove */
  onMove={(event: XRInteractionEvent) => ...}
>
  <Box />
</Interactive>

RayGrab

<RayGrab /> is a specialized <Interactive /> that can be grabbed and moved by controllers.

<RayGrab>
  <Box />
</RayGrab>

useInteraction

useInteraction subscribes an existing element to controller events.

The following interaction events are supported: onHover, onBlur, onSelect, onSelectEnd, onSelectStart, onSelectMissed, onSqueeze, onSqueezeEnd, onSqueezeStart, onSqueezeMissed, onMove.

const boxRef = useRef()
useInteraction(boxRef, 'onSelect', (event: XRInteractionEvent) => ...)

<Box ref={boxRef} />

useHitTest

Use this hook to perform a hit test for an AR environment. Also see XRHitTestResult.

useHitTest((hitMatrix: Matrix4, hit: XRHitTestResult) => {
  // use hitMatrix to position any object on the real life surface
  hitMatrix.decompose(mesh.position, mesh.quaternion, mesh.scale)
})

useXREvent

To handle controller events that are not bound to any object in the scene you can use useXREvent hook. This is a low-level abstraction that subscribes directly into the native XRInputSource (see XRInputSourceEvent).

useXREvent('squeeze', (event: XRControllerEvent) => ...)

It supports an optional third parameter with options for filtering by handedness.

useXREvent('squeeze', (event: XRControllerEvent) => ..., { handedness: 'left' | 'right' | 'none' })