perf(utils): applyBVH helper for opting scenes into accelerated raycasts - #372
Conversation
…casts
The SDK's per-frame interaction raycaster walks every triangle of
every mesh under xb.core.scene via THREE.Raycaster.intersectObject
(recursive). For demos with non-trivial scene geometry (e.g. portals
with 5 loaded immersive worlds), the per-triangle walk dominates the
main thread.
three-mesh-bvh's accelerated raycaster solves this but it has to be
opted into per geometry. Adds two public helpers:
- enableAcceleratedRaycast(): installs the THREE.Mesh prototype patch
and BufferGeometry helpers. Idempotent across callers (so multiple
subsystems can ping it).
- applyBVH(root, {recursive?}): walks the tree and builds a bounds
tree on every mesh's geometry. Demos opt in by calling this on the
root of their scene after meshes are added.
Geometries without a bounds tree continue to use the stock raycaster,
so flipping the prototype patch on globally is safe for existing code
that never calls applyBVH.
Also adds three-mesh-bvh to package.json + rollup externals so it
ships consistently across the SDK and addons. 6 new tests cover
prototype patch install, idempotency, recursive build, no-rebuild,
recursive=false stop, and dispose.
Portals demo loads 5 immersive scenes into xb.core.scene. The SDK's per-frame interaction raycast (Input.performRaycastOnScene) walks every triangle under the scene root each controller frame, and the per-triangle walk dominated the main thread in a perf trace before this. Calls xb.applyBVH(this) at the end of PortalGalleryScene.init() so the bounds tree is built once per geometry after all immersives are added. Subsequent raycasts go through three-mesh-bvh's accelerated path (O(log triangles) per ray instead of O(triangles)). Adds three-mesh-bvh to the demo importmap. Same version pinned by the SDK and other demos.
Drop the inline three-mesh-bvh prototype patches and import enableAcceleratedRaycast() from src/utils/BVHRaycast (added in google#372). The helper is idempotent across modules so other subsystems that also install BVH (objects_3d, portals) share the same patch without fighting over the prototype. This PR's branch is stacked on google#372 so the util is available; merge google#372 first.
per review feedback on google#367, three-mesh-bvh should be optional like troika-three-text is for TextView: type-only import at build time, dynamic import at runtime with try / catch + status tracking. apps without three-mesh-bvh installed (or without it in their importmap) keep working with the stock three.js raycaster. changes: - moved three-mesh-bvh from dependencies to devDependencies (still needed for the SDK's own build + tests, but not forced on consumers) - import type * as BVH from 'three-mesh-bvh' for compile-time types - enableAcceleratedRaycast() and applyBVH() are now async; they kick off the dynamic import on first call and share the same promise on subsequent calls - on import failure, both helpers log a one-line warn and return false (or no-op for applyBVH). raycasts fall back to the stock walker - new isBVHReady() sync check for consumers that want to know whether the patches are installed - disposeBVH() stays sync; no-op when BVH was never loaded tests updated to await the async helpers. 295 tests still passing. mirrors the dynamic-import pattern in src/ui/components/TextView.ts for troika-three-text.
Now that google#372's enableAcceleratedRaycast is async and three-mesh-bvh is an optional dynamic import, the per-detection computeBoundsTree call in getDepthMeshSnapshot needs to gate on whether the BVH module has actually loaded. When BVH isn't available (no install, no importmap), we skip the bounds-tree build and the stock raycaster handles the per-landmark intersections. Functionally a no-op for any consumer that already has three-mesh-bvh in their importmap (face_mirror demo).
Drop the inline three-mesh-bvh prototype patches and import enableAcceleratedRaycast() from src/utils/BVHRaycast (added in google#372). The helper is idempotent across modules so other subsystems that also install BVH (objects_3d, portals) share the same patch without fighting over the prototype. This PR's branch is stacked on google#372 so the util is available; merge google#372 first.
Now that google#372's enableAcceleratedRaycast is async and three-mesh-bvh is an optional dynamic import, the per-detection computeBoundsTree call in getDepthMeshSnapshot needs to gate on whether the BVH module has actually loaded. When BVH isn't available (no install, no importmap), we skip the bounds-tree build and the stock raycaster handles the per-landmark intersections. Functionally a no-op for any consumer that already has three-mesh-bvh in their importmap (face_mirror demo).
|
Awesome! We discussed BVH years ago in XR Blocks before releasing, but didn't end up with external demos, since while BVH should only be applied to dense, static environmental meshes (like the Some issues found by Gemini: Minor Issue to Consider (The if (obj instanceof THREE.Mesh && obj.geometry) {
// ... compute bounds tree ...
}Because Recommendation: You may want to skip computing the bounds tree for them to save time and memory: if (
obj instanceof THREE.Mesh &&
!(obj instanceof THREE.SkinnedMesh) &&
!(obj instanceof THREE.InstancedMesh) &&
obj.geometry
) { |
per review feedback on google#372, BVH isn't a silver bullet: tree build + memory cost, assumes static vertices, and the traversal can be slower than walking a few triangles for low-poly meshes. should only be applied to dense static environmental meshes, not globally. three changes: - applyBVH now skips THREE.SkinnedMesh and THREE.InstancedMesh. both override .raycast() on their own prototypes, so the BVH would never be consulted for them anyway. previously we'd pay the build cost for nothing. - applyBVH docstring now spells out the when/when-not. flags the dense-static rule and warns against global application. - demos/portals scopes the applyBVH call to each immersive sub-tree instead of the whole gallery root. the gallery root also holds portal discs, floating labels, the fade sphere, etc. which are low-poly / dynamic and don't benefit. 2 new tests cover the SkinnedMesh + InstancedMesh skip. 306 tests total (was 304).
Both subclasses override .raycast() on their own prototypes (not on THREE.Mesh.prototype), so the BVH path is never consulted for them. Building the bounds tree on them just wastes CPU + memory. 2 new tests cover the skip.
Calls out the dense-static rule + warns against global application. Useful guidance for future demo authors so the helper doesn't get sprayed at every scene root.
Previously called xb.applyBVH on the entire gallery root which also holds portal discs, floating labels, the fade sphere, etc. Those are low-poly / dynamic and don't benefit from BVH. Apply per immersive scene instead so we only pay the build cost on the dense static geometry the BVH is actually meant for.
d2987a9 to
942d971
Compare
Each immersive sub-tree contains exactly one shader skybox sphere with
`raycast = () => {}`, so the loop was building bounds trees no
raycaster ever consults: pure CPU/memory cost. Once an immersive gains
real raycastable geometry (loaded models, etc.), opt in again on those
specific meshes.
This reverts commit 787597d.
This reverts commit 942d971.
The original skip rationale ("InstancedMesh overrides .raycast() on its
own prototype so the BVH would never be consulted") was incorrect.
InstancedMesh.prototype.raycast (three.js v0.184.0,
src/objects/InstancedMesh.js:269-303) sets a shared internal
`_mesh.geometry = this.geometry` and calls `_mesh.raycast()` once per
instance. That call routes through the patched `Mesh.prototype.raycast`
(acceleratedRaycast), which short-circuits to `geometry.boundsTree`
whenever the shared geometry has one. So building a BVH on the shared
geometry accelerates every per-instance ray test.
Drops the InstancedMesh skip + the lock-in test, replaces with a test
that confirms the boundsTree IS built.
`THREE.BatchedMesh` is a `Mesh` subclass with its own raycast override
(three.js v0.184.0, src/objects/BatchedMesh.js:1382-1442) that iterates
per draw range and adjusts an internal mesh's drawRange before each
ray test.
three-mesh-bvh ships a dedicated pair for this case
(src/utils/ExtensionUtilities.js):
- `computeBatchedBoundsTree(geometryId)` builds per-draw-range BVHs
stored on `this.boundsTrees` (plural) on the BatchedMesh itself
- `acceleratedRaycast` has a separate `if (this.isBatchedMesh) ...`
branch that consults `boundsTrees` keyed by `geometryId`
Calling the standard `computeBoundsTree()` on the BatchedMesh's combined
geometry would index the full batched buffer (not per-draw-range), and
`acceleratedRaycast`'s BatchedMesh branch wouldn't find `boundsTrees`
either way. The standard helper is wrong for BatchedMesh.
Conservative skip until apps that need it call the batched helpers
directly. Doc + test included.
Single applyBVH call on a Group containing a regular Mesh + SkinnedMesh + InstancedMesh + BatchedMesh + plain Object3D, asserting each gets the right treatment in one pass: - Mesh → boundsTree built - SkinnedMesh → skipped - InstancedMesh → boundsTree built (per-instance raycast uses it) - BatchedMesh → skipped - Object3D → not a mesh, naturally untouched Locks the per-subclass filter behavior in as a single end-to-end test rather than relying solely on the isolated per-subclass tests above.
End-to-end raycast: build BVH on a PlaneGeometry mesh, fire raycaster.intersectObject through the patched Mesh.prototype.raycast, assert the hit data is correct. Catches regressions where the prototype patch silently breaks raycast results (e.g. if three-mesh-bvh's acceleratedRaycast signature ever changes, or the BufferGeometry helper hookup drifts). Cold-dispose safety: vi.resetModules() to force bvhProtoPatched back to false, then call disposeBVH(root) on a tree that never had applyBVH applied. Asserts no throw and no boundsTree side-effect. Exercises the early-return path on a cold module that the existing tests can't hit because module state persists across them.
thanks Ruofei! that makes sense as per the rationale and great points, yeah so have updated the docstring so hopefully it makes it a bit easier to know when to apply, dense/static/environmental only, not UI/primitives/dynamic. portals only applies it on its own script subtree, not skinned skip yep. on the instanced one though, i don't think gemini's right, added a test that asserts the per-subclass policy, that will hopefully prevent drifts for this part |
… THREE inline #372 landed the BVH helpers in the SDK (utils/BVHRaycast). drop the direct three-mesh-bvh import + manual Mesh/BufferGeometry prototype patch and call xb.enableAcceleratedRaycast() instead, which installs the same computeBoundsTree / disposeBoundsTree helpers via the SDK's dynamic import. the per-detect computeBoundsTree() / disposeBoundsTree() calls stay, they now resolve to the SDK-installed helpers. computeBoundsTree() is guarded since enableAcceleratedRaycast() is async (dynamic import), so an early Detect press before it resolves falls back to the stock raycaster instead of throwing. importmap keeps three-mesh-bvh (the SDK dynamic-imports it from the page's module graph at runtime), bumped to 0.9.10 to match the version the SDK is built against.
the SDK's per-frame interaction raycaster (
Input.performRaycastOnScene) walks every triangle of every mesh underxb.core.sceneviaTHREE.Raycaster.intersectObject(recursive). for demos with non-trivial scene geometry the per-triangle walk dominates the main thread. surfaced in a portals trace where the call chainintersectTriangle -> _computeIntersections -> raycast -> xb.intersectaccounted for ~550 ms over a 12 s window.three-mesh-bvh's accelerated raycaster solves this but has to be opted into per geometry. two public helpers added in
src/utils/BVHRaycast:enableAcceleratedRaycast(): installs theTHREE.Meshprototype patch +BufferGeometry.computeBoundsTreehelper. idempotent across callers so multiple subsystems can ping it safely.applyBVH(root, {recursive?}): walks the tree and builds a bounds tree on every mesh's geometry. demos opt in by calling this on the root of their scene after meshes are added.three-mesh-bvh is loaded as an optional dynamic import (devDependency, type-only build import,
await import('three-mesh-bvh')with try/catch). apps without it in their importmap don't break, they just don't get the accelerated raycast. both helpers are async and resolve tofalsein that case. same pattern as troika-three-text inTextView.applyBVHskipsSkinnedMesh(animated vertices invalidate the bind-pose bvh, and three'sSkinnedMesh.raycastoverrides the patched prototype anyway) andBatchedMesh(needs three-mesh-bvh's separatecomputeBatchedBoundsTreebuilding per-draw-range trees onthis.boundsTrees).InstancedMeshis kept in, its per-instance_mesh.raycast()does route through the patchedMesh.prototypeso the bvh on the shared geometry accelerates each instance test.geometries without a bounds tree continue using the stock raycaster, so flipping the prototype patch on globally is safe for existing code that never calls
applyBVH.opted in
demos/portalsby callingxb.applyBVH(this)at the end ofPortalGalleryScene.init()after all 5 immersive scenes are added.measured impact
portals demo trace before vs after (this PR + #371 both applied):
intersectTrianglegetVertexPositioncheckGeometryIntersection_computeIntersectionsWebGLRenderer.setSizegetProgramInfoLogidle went from 8% -> 60% of the trace window. massive frame-budget recovery on a demo that does interaction raycasting + occlusion.
12 tests cover the helpers: prototype patch install / idempotency, recursive build +
recursive=false, dispose, per-subclass skip filter (SkinnedMesh,BatchedMesh) +InstancedMeshbuild, a mixed-tree filter test (all 4 subclass types in oneapplyBVHcall, locks the policy in), end-to-end raycast through the patched path, and cold-dispose safety. 310 tests total.related: #367 (BVH for face_mirror's per-landmark raycasts; refactored to depend on this PR's util). #371 (occlusion pass setSize fix, contributes the setSize row above).