omnidotdev · coopbri · Nov 28, 2025 · Nov 25, 2025 · Nov 25, 2025 · Nov 25, 2025
diff --git a/.changeset/metal-cloths-deny.md b/.changeset/metal-cloths-deny.md
@@ -0,0 +1,26 @@
+---
+"@omnidotdev/rdk": minor
+---
+
+Added native WebXR support, powered by [`@react-three/xr`](https://github.com/pmndrs/xr).
+
+**BREAKING:** Removed `cameraSource` prop from XR component. Sessions now auto-configure themselves:
+
+```tsx
+// before
+<XR cameraSource="video">
+  <FiducialSession />
+</XR>
+
+// after
+<XR>
+  {/* auto-configures video mode */}
+  <FiducialSession />
+</XR>
+```
+
+**New Features:**
+
+- `ImmersiveSession` component for WebXR AR/VR
+- Nested `@react-three/xr`'s store nested under `useXRStore`'s `immersive` property
+- Added `ImmersiveMode` type export, which maps to and from [official WebXR modes](https://www.w3.org/TR/webxr/#xrsessionmode-enum) (`immersive-ar` ↔ `ar`, `immersive-vr` ↔ `vr`, `inline` ↔ `inline`)
diff --git a/README.md b/README.md
@@ -11,27 +11,26 @@
 
 > [!IMPORTANT]
 > **Project Status:** 🚧 This project is **brand new**.
-> Currently, fiducial marker-based AR via [AR.js](https://github.com/ar-js-org/ar.js) and location-based AR via [LocAR.js](https://github.com/ar-js-org/locar.js) are working (see [`apps/`](./apps/) for demos), though they are **experimental**. Contributions (PRs, [Omni organization sponsorship](https://github.com/sponsors/omnidotdev)) appreciated.
-> Native WebXR integration via `@react-three/xr` is coming next.
+> Currently, fiducial marker-based AR via [AR.js](https://github.com/ar-js-org/ar.js), location-based AR via [LocAR.js](https://github.com/ar-js-org/locar.js), and WebXR via [`@react-three/xr`](https://github.com/pmndrs/xr) are working (see [`apps/`](./apps/) for demos), though they are **experimental**. Contributions (PRs, [Omni organization sponsorship](https://github.com/sponsors/omnidotdev)) appreciated.
 
 ## Overview
 
-RDK unifies multiple XR technologies, such as AR.js for marker-based AR, LocAR.js for geolocation-based AR, and WebXR (coming soon) for device-native support under one React-first abstraction powered by Three.js.
-
-| Capability/Use Case                  | Status          | Backend (Current or Proposed)                                                                        | Android | iOS | Notes                                                                                     |
-| ------------------------------------ | --------------- | ---------------------------------------------------------------------------------------------------- | ------- | --- | ----------------------------------------------------------------------------------------- |
-| **Fiducial (Pattern/Barcode)**       | ⚗️ Experimental | [AR.js (ARToolKit)](https://github.com/ar-js-org/ar.js)                                              | ✅      | ✅  | Uses `.patt` or barcode markers. Reliable for printed markers. No WebXR dependency.       |
-| **Image Tracking (Natural Feature)** | 🧭 Planned      | [AR.js (ARToolKit)](https://github.com/ar-js-org/ar.js)                                              | N/A     | N/A | May use `.mind` or `XRTrackedImage`. Ideal for logos or posters. Requires image database. |
-| **Geolocation / World Anchors**      | ⚗️ Experimental | [LocAR.js](https://github.com/ar-js-org/locar.js)                                                    | ✅      | ✅  | Uses GPS + compass; may later integrate Mapbox or Cesium.                                 |
-| **WebXR Native AR/VR Session**       | 🧭 Planned      | [`@react-three/xr`](https://github.com/pmndrs/xr)                                                    | N/A     | N/A | Entry point for true AR/VR sessions. Ties into `XRSessionProvider`.                       |
-| **Face Tracking**                    | 🧭 Planned      | -                                                                                                    | N/A     | N/A | Uses webcam + ML model; lightweight and fast.                                             |
-| **Body/Pose Tracking**               | 🧭 Planned      | [WebXR Body Tracking](https://github.com/immersive-web/body-tracking)                                | N/A     | N/A | Real-time skeletal tracking. GPU/WebGL acceleration required.                             |
-| **Hand Tracking**                    | 🧭 Planned      | -                                                                                                    | N/A     | N/A | Supported on Chrome + Meta; ML fallback possible.                                         |
-| **Plane/Surface Detection**          | 🧭 Planned      | [WebXR Hit Test API](https://immersive-web.github.io/hit-test)/ar.js (limited)                       | N/A     | N/A | Enables AR object placement on flat surfaces.                                             |
-| **Depth Sensing/Environment Mesh**   | 🧭 Planned      | [WebXR Depth Sensing API](https://immersive-web.github.io/depth-sensing)                             | N/A     | N/A | Provides per-pixel depth; early spec.                                                     |
-| **SLAM/Visual Positioning (VPS)**    | 🧭 Planned      | Custom                                                                                               | N/A     | N/A | Requires world map data; long-term goal.                                                  |
-| **Voice/Gesture Interaction**        | 🧭 Planned      | [Web Speech API](https://developer.mozilla.org/en-US/docs/Web/API/Web_Speech_API)/MediaPipe Gestures | N/A     | N/A | Enables multimodal input: voice, hand, gaze.                                              |
-| **Mixed Reality Compositing**        | 🧭 Planned      | WebXR Layers/CanvasCaptureStream                                                                     | N/A     | N/A | Transparent overlays/live compositing.                                                    |
+RDK unifies multiple spatial and XR technologies, such as AR.js for marker-based AR, LocAR.js for geolocation-based AR, and WebXR for device-native support under one React-first abstraction powered by Three.js.
+
+| Capability/Use Case                  | Status          | Backend (Current or Proposed)                                                                        | Android | iOS            | Notes                                                                                     |
+| ------------------------------------ | --------------- | ---------------------------------------------------------------------------------------------------- | ------- | -------------- | ----------------------------------------------------------------------------------------- |
+| **Fiducial (Pattern/Barcode)**       | ⚗️ Experimental | [AR.js (ARToolKit)](https://github.com/ar-js-org/ar.js)                                              | ✅      | ✅             | Uses `.patt` or barcode markers. Reliable for printed markers. No WebXR dependency.       |
+| **Image Tracking (Natural Feature)** | 🧭 Planned      | [AR.js (ARToolKit)](https://github.com/ar-js-org/ar.js)                                              | N/A     | N/A            | May use `.mind` or `XRTrackedImage`. Ideal for logos or posters. Requires image database. |
+| **Geolocation / World Anchors**      | ⚗️ Experimental | [LocAR.js](https://github.com/ar-js-org/locar.js)                                                    | ✅      | ✅             | Uses GPS + compass; may later integrate Mapbox or Cesium.                                 |
+| **WebXR Native AR/VR Session**       | ⚗️ Experimental | [`@react-three/xr`](https://github.com/pmndrs/xr)                                                    | ✅      | ❌[^ios-webxr] | Entry point for immersive AR/VR sessions.                                                 |
+| **Face Tracking**                    | 🧭 Planned      | -                                                                                                    | N/A     | N/A            | Uses webcam + ML model; lightweight and fast.                                             |
+| **Body/Pose Tracking**               | 🧭 Planned      | [WebXR Body Tracking](https://github.com/immersive-web/body-tracking)                                | N/A     | N/A            | Real-time skeletal tracking. GPU/WebGL acceleration required.                             |
+| **Hand Tracking**                    | 🧭 Planned      | -                                                                                                    | N/A     | N/A            | Supported on Chrome + Meta; ML fallback possible.                                         |
+| **Plane/Surface Detection**          | 🧭 Planned      | [WebXR Hit Test API](https://immersive-web.github.io/hit-test)/ar.js (limited)                       | N/A     | N/A            | Enables AR object placement on flat surfaces.                                             |
+| **Depth Sensing/Environment Mesh**   | 🧭 Planned      | [WebXR Depth Sensing API](https://immersive-web.github.io/depth-sensing)                             | N/A     | N/A            | Provides per-pixel depth; early spec.                                                     |
+| **SLAM/Visual Positioning (VPS)**    | 🧭 Planned      | Custom                                                                                               | N/A     | N/A            | Requires world map data; long-term goal.                                                  |
+| **Voice/Gesture Interaction**        | 🧭 Planned      | [Web Speech API](https://developer.mozilla.org/en-US/docs/Web/API/Web_Speech_API)/MediaPipe Gestures | N/A     | N/A            | Enables multimodal input: voice, hand, gaze.                                              |
+| **Mixed Reality Compositing**        | 🧭 Planned      | WebXR Layers/CanvasCaptureStream                                                                     | N/A     | N/A            | Transparent overlays/live compositing.                                                    |
 
 ## Demos
 
@@ -83,7 +82,11 @@ Install RDK and required peer dependencies:
 bun add @omnidotdev/rdk @ar-js-org/ar.js @react-three/fiber locar react react-dom three
 ```
 
-See [`apps/fiducial-demo`](./apps/fiducial-demo) for an example of usage. More demos will be added as more use cases beyond fiducial marker-based AR are implemented.
+See the demo applications for examples of usage:
+
+- [`apps/fiducial-demo`](./apps/fiducial-demo): Fiducial marker tracking powered by AR.js
+- [`apps/geolocation-demo`](./apps/geolocation-demo): GPS-based AR powered by LocAR.js
+- [`apps/immersive-demo`](./apps/immersive-demo): WebXR powered by `@react-three/xr`
 
 ## Goals: the "Why"
 
@@ -119,3 +122,5 @@ See Omni's [contributing docs](https://docs.omni.dev/contributing/overview).
 ## License
 
 The code in this repository is licensed under MIT, &copy; Omni LLC. See [LICENSE.md](LICENSE.md) for more information.
+
+[^ios-webxr]: [iOS does not currently natively support WebXR](https://caniuse.com/webxr), but an iOS fallback is planned for the RDK immersive module.
diff --git a/Tiltfile b/Tiltfile
@@ -22,6 +22,14 @@ local_resource(
   labels=["demo-geolocation"]
 )
 
+local_resource(
+  "dev-demo-immersive",
+  serve_cmd="bun dev --filter rdk-immersive-demo",
+  auto_init=False,
+  trigger_mode=TRIGGER_MODE_MANUAL,
+  labels=["demo-immersive"]
+)
+
 local_resource(
   "build",
   "bun run build",

diff --git a/apps/fiducial-demo/package.json b/apps/fiducial-demo/package.json
@@ -9,17 +9,16 @@
     "lint": "biome check .",
     "lint:fix": "biome check --write .",
     "format": "biome format --write .",
-    "type-check": "tsc --noEmit",
     "clean": "rm -rf build"
   },
   "devDependencies": {
-    "@biomejs/biome": "^2.3.2",
-    "@types/react": "^19.2.2",
-    "@types/react-dom": "^19.2.2",
-    "@types/three": "^0.180.0",
-    "@vitejs/plugin-react": "^5.1.0",
+    "@biomejs/biome": "^2.3.8",
+    "@types/react": "^19.2.7",
+    "@types/react-dom": "^19.2.3",
+    "@types/three": "^0.181.0",
+    "@vitejs/plugin-react": "^5.1.1",
     "typescript": "^5.9.3",
-    "vite": "^7.1.12",
+    "vite": "^7.2.4",
     "vite-plugin-mkcert": "^1.17.9",
     "vite-tsconfig-paths": "^5.1.4"
   },
@@ -29,6 +28,6 @@
     "@react-three/fiber": "^9.4.0",
     "react": "^19.2.0",
     "react-dom": "^19.2.0",
-    "three": "^0.180.0"
+    "three": "^0.181.2"
   }
 }
diff --git a/apps/fiducial-demo/src/App.tsx b/apps/fiducial-demo/src/App.tsx
@@ -16,7 +16,7 @@ const App = () => (
     <hemisphereLight intensity={0.6} />
     <directionalLight position={[5, 5, 5]} intensity={1} />
 
-    <XR cameraSource="video">
+    <XR>
       <FiducialSession>
         <FiducialAnchor params={{ smooth: true }} patternUrl="/data/rdk.patt">
           <Supertorus

diff --git a/apps/geolocation-demo/package.json b/apps/geolocation-demo/package.json
@@ -9,26 +9,25 @@
     "lint": "biome check .",
     "lint:fix": "biome check --write .",
     "format": "biome format --write .",
-    "type-check": "tsc --noEmit",
     "clean": "rm -rf build"
   },
   "devDependencies": {
-    "@biomejs/biome": "^2.3.2",
-    "@types/react": "^19.2.2",
-    "@types/react-dom": "^19.2.2",
-    "@types/three": "^0.180.0",
-    "@vitejs/plugin-react": "^5.1.0",
+    "@biomejs/biome": "^2.3.8",
+    "@types/react": "^19.2.7",
+    "@types/react-dom": "^19.2.3",
+    "@types/three": "^0.181.0",
+    "@vitejs/plugin-react": "^5.1.1",
     "typescript": "^5.9.3",
-    "vite": "^7.1.12",
+    "vite": "^7.2.4",
     "vite-plugin-mkcert": "^1.17.9",
     "vite-tsconfig-paths": "^5.1.4"
   },
   "dependencies": {
     "@omnidotdev/rdk": "workspace:*",
     "@react-three/fiber": "^9.4.0",
-    "locar": "^0.1.4",
+    "locar": "^0.1.5",
     "react": "^19.2.0",
     "react-dom": "^19.2.0",
-    "three": "^0.180.0"
+    "three": "^0.181.2"
   }
 }
diff --git a/apps/geolocation-demo/src/App.tsx b/apps/geolocation-demo/src/App.tsx
@@ -29,7 +29,7 @@ const App = () => (
     <directionalLight position={[10, 10, 10]} intensity={2} castShadow />
     <directionalLight position={[-10, 10, -10]} intensity={1} />
 
-    <XR cameraSource="video">
+    <XR>
       <GeolocationSession
         options={
           {

diff --git a/apps/immersive-demo/README.md b/apps/immersive-demo/README.md
@@ -0,0 +1,19 @@
+# RDK Immersive XR Demo
+
+This demo showcases RDK's immersive XR capabilities powered by WebXR and `@react-three/xr`.
+
+## Usage
+
+### Development
+
+```bash
+bun install
+bun dev
+```
+
+### Production Build
+
+```bash
+bun run build
+bun preview
+```
diff --git a/apps/immersive-demo/index.html b/apps/immersive-demo/index.html
@@ -0,0 +1,31 @@
+<!DOCTYPE html>
+<html lang="en">
+    <head>
+    	<meta charset="UTF-8" />
+    	<meta name="viewport" content="width=device-width, initial-scale=1.0" />
+
+    	<title>RDK Immersive XR Demo</title>
+
+    	<style>
+    		body {
+    			width: 100%;
+    			height: 100vh;
+    			margin: 0;
+    			padding: 0;
+    			overflow: hidden;
+    			font-family: system-ui, -apple-system, sans-serif;
+    		}
+
+    		#root {
+    			width: 100%;
+    			height: 100%;
+    		}
+    	</style>
+    </head>
+
+    <body>
+    	<div id="root"></div>
+
+    	<script type="module" src="/src/index.tsx"></script>
+    </body>
+</html>
diff --git a/apps/immersive-demo/package.json b/apps/immersive-demo/package.json
@@ -0,0 +1,34 @@
+{
+  "name": "rdk-immersive-demo",
+  "private": true,
+  "type": "module",
+  "scripts": {
+    "dev": "vite",
+    "build": "vite build",
+    "preview": "vite preview",
+    "lint": "biome check .",
+    "lint:fix": "biome check --write .",
+    "format": "biome format --write .",
+    "clean": "rm -rf build"
+  },
+  "devDependencies": {
+    "@biomejs/biome": "2.3.8",
+    "@types/react": "^19.2.7",
+    "@types/react-dom": "^19.2.3",
+    "@types/three": "^0.181.0",
+    "@vitejs/plugin-react": "^5.1.1",
+    "typescript": "~5.9.3",
+    "vite": "^7.2.4",
+    "vite-plugin-mkcert": "^1.17.9",
+    "vite-tsconfig-paths": "^5.1.4"
+  },
+  "dependencies": {
+    "@omnidotdev/rdk": "workspace:*",
+    "@react-three/drei": "^10.7.7",
+    "@react-three/fiber": "^9.4.0",
+    "@react-three/xr": "^6.6.28",
+    "react": "^19.2.0",
+    "react-dom": "^19.2.0",
+    "three": "^0.181.2"
+  }
+}