React components and hooks for creating VR/AR applications with @react-three/fiber
npm install @react-three/xr
These demos are real, you can click them! They contain the full code, too.
The following adds a button to start your session and controllers inside an XR manager to prepare your scene for WebXR rendering and interaction.
import { VRButton, ARButton, XR, Controllers, Hands } from '@react-three/xr'
import { Canvas } from '@react-three/fiber'
function App() {
return (
<>
<VRButton />
<Canvas>
<XR>
<Controllers />
<Hands />
<mesh>
<boxGeometry />
<meshBasicMaterial color="blue" />
</mesh>
</XR>
</Canvas>
</>
)
}<XRButton /> is an HTML <button /> that can be used to init and display info about your WebXR session. This is aliased by ARButton and VRButton with sensible session defaults.
<XRButton
/* The type of `XRSession` to create */
mode={'AR' | 'VR' | 'inline'}
/**
* `XRSession` configuration options
* @see https://immersive-web.github.io/webxr/#feature-dependencies
*/
sessionInit={{ optionalFeatures: ['local-floor', 'bounded-floor', 'hand-tracking', 'layers'] }}
/** Whether this button should only enter an `XRSession`. Default is `false` */
enterOnly={false}
/** Whether this button should only exit an `XRSession`. Default is `false` */
exitOnly={false}
>
{/* Can accept regular DOM children and has an optional callback with the XR button status (unsupported, exited, entered) */}
{(status) => `WebXR ${status}`}
</XRButton><XR /> is a WebXR manager that configures your scene for XR rendering and interaction. This lives within a R3F <Canvas />.
<Canvas>
<XR
/**
* Enables foveated rendering. Default is `0`
* 0 = no foveation, full resolution
* 1 = maximum foveation, the edges render at lower resolution
*/
foveation={0}
/** Type of WebXR reference space to use. Default is `local-floor` */
referenceSpace="local-floor"
/** Called as an XRSession is requested */
onSessionStart={(event: XREvent<XRManagerEvent>) => ...}
/** Called after an XRSession is terminated */
onSessionEnd={(event: XREvent<XRManagerEvent>) => ...}
/** Called when an XRSession is hidden or unfocused. */
onVisibilityChange={(event: XREvent<XRSessionEvent>) => ...}
/** Called when available inputsources change */
onInputSourcesChange={(event: XREvent<XRSessionEvent>) => ...}
>
{/* All your regular react-three-fiber elements go here */}
</XR>
</Canvas>This hook gives you access to the current XRState configured by <XR />.
const {
// An array of connected `XRController`
controllers,
// Whether the XR device is presenting in an XR session
isPresenting,
// Whether hand tracking inputs are active
isHandTracking,
// A THREE.Group representing the XR viewer or player
player,
// The active `XRSession`
session,
// `XRSession` foveation. This can be configured as `foveation` on <XR>. Default is `0`
foveation,
// `XRSession` reference-space type. This can be configured as `referenceSpace` on <XR>. Default is `local-floor`
referenceSpace
} = useXR()To subscribe to a specific key, useXR accepts a Zustand selector:
const player = useXR((state) => state.player)Controllers can be added with <Controllers /> for motion-controllers and/or <Hands /> for hand-tracking. These will activate whenever their respective input mode is enabled on-device and provide live models for a left and right XRController.
<Controllers
/** Optional material props to pass to controllers' ray indicators */
rayMaterial={{ color: 'blue' }}
/** Whether to hide controllers' rays on blur. Default is `false` */
hideRaysOnBlur={false}
/>
<Hands
// Optional custom models per hand. Default is the Oculus hand model
modelLeft="/model-left.glb"
modelRight="/model-right.glb"
/>useController references an XRController by handedness, exposing position and orientation info.
const leftController = useController('left')
const rightController = useController('right')
const gazeController = useController('none')XRController is an Object3D that represents an XRInputSource with the following properties:
index: number
controller: THREE.XRTargetRaySpace
grip: THREE.XRGripSpace
hand: THREE.XRHandSpace
inputSource: XRInputSourceTo interact with objects using controllers you can use <Interactive /> component or useInteraction hook. They allow adding controller event handlers to your objects.
<Interactive /> wraps your objects and accepts XR controller event handlers as props. Supports select, hover, blur and squeeze events (see XR inputsources).
<Interactive
/* Called when hovered by a controller */
onHover={(event: XRInteractionEvent) => ...}
/* Called when unhovered by a controller */
onBlur={(event: XRInteractionEvent) => ...}
/* Called on button press when selected by a controller */
onSelectStart={(event: XRInteractionEvent) => ...}
/* Called on button release when selected by a controller */
onSelectEnd={(event: XRInteractionEvent) => ...}
/* Called on button release when another interactive is selected by a controller */
onSelectMissed={(event: XRInteractionEvent) => ...}
/* Called when selected by a controller */
onSelect={(event: XRInteractionEvent) => ...}
/* Called on button press when squeezed by a controller */
onSqueezeStart={(event: XRInteractionEvent) => ...}
/* Called on button release when squeezed by a controller */
onSqueezeEnd={(event: XRInteractionEvent) => ...}
/* Called on button release when another interactive is squeezed by a controller */
onSqueezeMissed={(event: XRInteractionEvent) => ...}
/* Called when squeezed by a controller */
onSqueeze={(event: XRInteractionEvent) => ...}
/* Called when a controller moves over the object, equivalent to pointermove */
onMove={(event: XRInteractionEvent) => ...}
>
<Box />
</Interactive><RayGrab /> is a specialized <Interactive /> that can be grabbed and moved by controllers.
<RayGrab>
<Box />
</RayGrab>useInteraction subscribes an existing element to controller events.
The following interaction events are supported: onHover, onBlur, onSelect, onSelectEnd, onSelectStart, onSelectMissed, onSqueeze, onSqueezeEnd, onSqueezeStart, onSqueezeMissed, onMove.
const boxRef = useRef()
useInteraction(boxRef, 'onSelect', (event: XRInteractionEvent) => ...)
<Box ref={boxRef} />Use this hook to perform a hit test for an AR environment. Also see XRHitTestResult.
useHitTest((hitMatrix: Matrix4, hit: XRHitTestResult) => {
// use hitMatrix to position any object on the real life surface
hitMatrix.decompose(mesh.position, mesh.quaternion, mesh.scale)
})To handle controller events that are not bound to any object in the scene you can use useXREvent hook. This is a low-level abstraction that subscribes directly into the native XRInputSource (see XRInputSourceEvent).
useXREvent('squeeze', (event: XRControllerEvent) => ...)It supports an optional third parameter with options for filtering by handedness.
useXREvent('squeeze', (event: XRControllerEvent) => ..., { handedness: 'left' | 'right' | 'none' })