feat(events)!: pointerevents manager and state #529

garrlker · 2024-01-31T04:54:07Z

Events

Event Details

Features

Event support for Primitives
Event Bubbling and Propagation
- Raycast Propogation
- Hierarchy Propogation
Events are emitted off of TresCanvas
Forced Raycasts
Duplicate Checks (no event handler called twice)

Chores

Getting up to date with the v4 branch
Documentation
Unit tests

netlify · 2024-01-31T04:54:12Z

✅ Deploy Preview for tresjs-docs ready!

Name	Link
🔨 Latest commit	`580029d`
🔍 Latest deploy log	https://app.netlify.com/sites/tresjs-docs/deploys/65bc7351e16912000880f427
😎 Deploy Preview	https://deploy-preview-529--tresjs-docs.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

playground/src/pages/raycaster/Propogation.vue

andretchen0

@garrlker

Looks like events are coming along! Great!

I'm not overly familiar with Tres internals, so this review is taking me a while. I'll have another look tomorrow, but I'll go ahead and submit what I've got so you can take a look when it's convenient.

playground/src/pages/raycaster/Propogation.vue

andretchen0 · 2024-01-31T22:25:33Z

src/composables/useRaycaster/index.ts

@@ -43,10 +48,10 @@ export const useRaycaster = (

    raycaster.value.setFromCamera(new Vector2(x, y), camera.value)

-    return raycaster.value.intersectObjects(objects.value, false)
+    return raycaster.value.intersectObjects(objects.value, true)


If I understand correctly, this will test against all children of the scene and recurse down through all children.

If that's right, is that worth optimizing in this PR? Or ever?

For context, it looks like R3F does some optimization to keep some eventless objects from being tested for intersections.

Correct, this currently tests against all children of the scene recursively.

I'm not against optimizing the events in this PR, but overall our CPU usage is somehow lower on my system than the main

The reason we need to include the entire scene in the intersectObjects() is because of the event propagation.
Children that do not have events are still the starting point for parents that do.

There are a few optimizations I've thought of since starting this

Only include node's with events and their direct descendants in the hit test. Being a custom renderer gives us this flexibility to do this on scene insertion. The original solution used a similar idea to only include node's having events and blocking-objects

Add the .self event modifier so a node will only listen to their own events. That let's us exclude any children of that node without events from the hit test as well

Add a no-events attribute that remove that object and possibly it's children from the hit test as well.

I think we're calling intersectObjects() way more than needed via the getIntersects() call. Realistically that should only need to be called on pointer-move, than any other event just grab the intersect results from there. Though I'm still not sure that are actually doing a hit test on every single event. Need to debug it more.

Sounds good!

It sounds like there's plenty to think about and discuss with the others. I don't want to keep these events out of Tres. So maybe we open another PR when this PR is merged.

Only include node's with events and their direct descendants in the hit test. Being a custom renderer gives us this flexibility to do this on scene insertion.

Yeah, something like that is what I had in mind as well. We'd have to watch event updates as well.

After talking with @alvarosabu and @Tinoooo it looks like we have enough time before the V4 release for me to take a stab at optimizing this some more

I was able to confirm intersectObjects is being called multiple times per frame (and even per event).
I've simplified it's usage. I'm seeing about half the cpu % in my test scenes.

I think we could add a flag to the localstate __tres of the instances marking if we expect it to have events to optimize the traversing of the scene. See recently merged #522

Meshes and groups:

instance.__tres.events = true

Dunno, materials and geometries:

instance.__tres.events = false

Ahh good idea. The localstate __tres will be very helpful

Thank you for calling it out!

I've added the EventManager onto Tres's Context

Let me know if that was what you had in mind

new file: playground/src/pages/raycaster/Propogation.vue * Started work on interactive Event Propogation playground example modified: src/components/TresCanvas.vue * Import and use `useEventStore` * defineEmits for all expected pointer events so we may emit propogated events off of the canvasa modified: src/composables/index.ts new file: src/composables/useEventStore/index.ts * Started work on an event store. I'm not sure this counts as a store just yet * Wired up majority of pointer events * Added event propogation * Does not require using userData scene props or nodeOps for registering objects to scene modified: src/composables/useRaycaster/index.ts * Added new event listeners to power newly supported pointer events. We now check whole scene/children when calling intersectObjects. * Created new EventHooks for new events * Added `forceUpdate` function that allows for pointer-move events to work without mouth movement (good for when camera is moving but mouse is not) modified: src/core/nodeOps.ts * Added supported events to array so they don't get received as props * (temporarily) unhook current pointer event solution to iterate on useEventStore modified: src/utils/index.ts * Added Camel-to-kebab case util

andretchen0 · 2024-02-02T01:45:47Z

src/composables/useEventStore/index.ts

+          let parent = object.parent
+          while(parent !== null && !stopPropagating) {
+            executeEventListeners(parent[eventName], intersection, event)
+            parent = parent.parent


One click should probably cause no more than one call to a given click handler.

For a given click, etc., a particular click handler shouldn't be called more than once, it seems to me. I think the way it works currently will be confusing in "real" world scenarios.

Currently

This shows the result of a single click with the current bubbling setup:

The console shows that the box at the top of the pyramid gets its click handler called 4 times.

"Realer" world

The above doesn't seem wrong to me, but I was trying to think about it as a Tres user and I think it breaks down in "realer" world cases.

Here's a simple example: buy a bicycle.

<Bicycle @click="buy"> <Frame> <Wheel /> <Wheel /> </Frame> </Bicycle>

Positioning the canvas at a certain angle, you could click both wheels and the frame, and you'll buy 3 bicycles instead of 1.

R3F removes duplicates

I checked out R3F. They remove duplicates.

Here's a StackBlitz with a simple setup like your playground.

Great catch!

I will be sure to add duplicate checks somehow

The latest changes include duplicate checks and it's working pretty well

Thanks for catching that!

… to work while multiple TresCanvas' are being used

…moving intersects into a ref that is updated on pointer-move

alvarosabu · 2024-02-02T09:11:50Z

@garrlker Regarding the store, I would suggest adding it to the TresContext directly instead of creating a separate store, that way is available on all instances via localState

andretchen0 · 2024-03-09T13:20:51Z

@alvarosabu

Regarding the store

As an aside, is there any interest in using something like Pinia for the store/context? Probably after v4?

Context

R3F uses their own reactive Pinia-like store (zustand) for similar purposes.

andretchen0 · 2024-03-09T14:29:05Z

Meta

@garrlker Regarding the store, I would suggest adding it to the TresContext directly instead of creating a separate store, that way is available on all instances via localState

@alvarosabu , that answers this question from @garrlker 's source, right?

// QUESTION: Maybe I should move the useEventStore logic into usePointerEventHandler? I seem to have created a similar pattern
// usePointerEventHandler({ scene: scene.value, contextParts: context.value })

playground/src/pages/raycaster/Propagation.vue

src/composables/useEventStore/index.ts

garrlker · 2024-03-17T23:40:11Z

As of right now there's no way to replace it without modifying the core but I think we could support that long term and not need too many changes

alvarosabu · 2024-03-19T16:00:23Z

Ohhh @garrlker this already looks great. I just tested the demos and the different options, propagation, and especially ev.stopPropagation() are working as I would expect them. Great job!

I think we can improve the Event object that we are returning. Comparing quickly the event returned by clicking (left R3F vs right Vue):

For example, returning and adding type of Event would be important:

{
  type: "click",
  distance: ...
}

For example here, I am trying to stop the propagation of the Boxes on the playground, since the color was pass as a parameter now I would need to decide the color programmatically inside of the click, so having info about the type of pointer could be good

wdyt @garrlker , @andretchen0 do you see any other property that could be important for the end user?

andretchen0 · 2024-03-19T17:39:37Z

wdyt @garrlker , @andretchen0 do you see any other property that could be important for the end user?

Here's the R3F approach in a nutshell:

Events contain the browser event as well as the three.js event data (object, point, distance, etc).

https://docs.pmnd.rs/react-three-fiber/api/events

That seems like a good approach to me.

garrlker · 2024-03-20T04:08:24Z

Great catch!

ngl I've never liked how verbose R3F events can be, mostly because I like to log the whole events in my console when debugging. But that's more personal preference 😅

I've started working on adding more properties to the event details object.

Also, I've added my personal todo list to the PR so it's easier to see what's been implemented. If you think anythings missing feel free to add more.

alvarosabu · 2024-03-20T08:36:36Z

Hey @garrlker I just updated the branch to latest v4, everything smooth,I'm also gonna take over docs

Getting up to date with the v4 branch
Documentation

Regarding:

I'm going to push the perf optimization work into a new issue for later so it won't hold this up. The perf is still better than before so I don't think it's high priority.

We can iterate in a follow up task after v4

alvarosabu · 2024-03-22T09:46:33Z

@garrlker Doing testing of each event I realise that the pointer-missed returns an empty array

I checked it on R3F and you get an event object similar to the other events with the type of event that you missed. Could we have a similar behavior?

andretchen0 · 2024-03-22T14:08:19Z

Tested with primitives. Works! Great! 👍

alvarosabu · 2024-03-22T14:17:23Z

@andretchen0 that was my next test, thanks for taking care of it 💚

andretchen0 · 2024-03-22T23:12:23Z

@andretchen0 that was my next test, thanks for taking care of it 💚

No problem. 😉

Just to be clear, I tested events added on the <primitive />, not added to the THREE.Mesh in <script setup>, for example.

Fwiw, here's the Vue file I set up if that's handy for anyone for double-checking primitives or my write-up.

<script setup lang="ts">
import type { ThreeEvent } from '@tresjs/core'
import { TresCanvas } from '@tresjs/core'
import { BasicShadowMap, SRGBColorSpace, NoToneMapping, BoxGeometry, MeshToonMaterial, Mesh, MathUtils } from 'three'
import { TresLeches, useControls } from '@tresjs/leches'
import { OrbitControls } from '@tresjs/cientos'
import '@tresjs/leches/styles'

const gl = {
  clearColor: '#202020',
  shadows: true,
  alpha: false,
  shadowMapType: BasicShadowMap,
  outputColorSpace: SRGBColorSpace,
  toneMapping: NoToneMapping,
}

const { stopPropagation } = useControls({
  stopPropagation: false,
})

function onClick(ev: ThreeEvent<MouseEvent>) {
  console.log('click', ev)
  if (stopPropagation.value) ev.stopPropagation()
  ev.object.material.color.set('#008080')
}

function onDoubleClick(ev: ThreeEvent<MouseEvent>) {
  console.log('double-click', ev)
  if (stopPropagation.value) ev.stopPropagation()
  ev.object.material.color.set('#FFD700')
}

function onPointerEnter(ev: ThreeEvent<MouseEvent>) {
  if (stopPropagation.value) ev.stopPropagation()
  ev.object.material.color.set('#CCFF03')
}

function onPointerLeave(ev: ThreeEvent<MouseEvent>) {
  if (stopPropagation.value) ev.stopPropagation()
  /*  ev.object.material.color.set('#efefef') */
}

function onPointerMove(ev: ThreeEvent<MouseEvent>) {
  if (stopPropagation.value) ev.stopPropagation()
}

function onContextMenu(ev: ThreeEvent<MouseEvent>) {
  console.log('context-menu', ev)
  if (stopPropagation.value) ev.stopPropagation()
  ev.object.material.color.set('#FF4500')
}

function onPointerMissed(ev: ThreeEvent<MouseEvent>) {
  console.log('pointer-missed', ev)
  if (stopPropagation.value) ev.stopPropagation()
}

const SIDE = 3
const COUNT = SIDE * SIDE * SIDE
const SPREAD = 6
const [LO, HI] = [-SPREAD * 0.5, SPREAD * 0.5]
const lerp = MathUtils.lerp

const geo = new BoxGeometry()
const objs = new Array(COUNT).fill(0).map((_, i) => {
  const mesh = new Mesh(geo, new MeshToonMaterial({ color: '#efefef' }))
  mesh.position.set(
    lerp(LO, HI, (i % SIDE) / (SIDE - 1)), 
    lerp(LO, HI, (Math.floor(i / SIDE) % SIDE) / (SIDE - 1)),
    lerp(LO, HI, Math.floor(i / (SIDE * SIDE)) / (SIDE - 1)),
  )
  return mesh
})
</script>

<template>
  <TresLeches />
  <TresCanvas
    window-size
    v-bind="gl"
  >
    <TresPerspectiveCamera :position="[10, 10, 10]" />
    <OrbitControls />
    <primitive
      v-for="obj, i in objs"
      :key="i"
      :object="obj"
      @click="onClick"
      @double-click="onDoubleClick"
      @pointer-enter="onPointerEnter"
      @pointer-leave="onPointerLeave"
      @pointer-move="onPointerMove"
      @context-menu="onContextMenu"
      @pointer-missed="onPointerMissed"
    />
    <TresDirectionalLight :intensity="1" />
    <TresAmbientLight :intensity="1" />
  </TresCanvas>
</template>

…aterial color. Add ability to force event system updates even when mouse hasn't moved. Enhance pointer-enter/leave events. Update types Box.vue * Added pointer-missed handler * set the materials flash color using the object coming off of the event instead of a ref UseRaycaster * Flesh out event details to include * all mouse event properties * intersections * tres camera * camera raycaster * source event * mouse position delta * stopPropagating stub * and unprojectedPoint (this needs work, cant get the math to work) UseTresContextProvider * Add TresEventManager type to TresContext useTresEventManager * Add forceUpdate method to allow apps to force an event system update even when the mouse hasnt moved * Add pointerMissed event * Properly implement pointer-enter/pointer-leave events * Before now, pointer-enter | leave were only called on first object in intersection, now we execute the events for all entered/left objects * Use stopPropagating property included on event object

…es into 515-pointer-eventmanager-state

garrlker · 2024-04-16T06:08:36Z

Hey everyone, I've pushed up my latest. This includes a few fixes from the previous comments and some other features

Most notably it includes a much more detailed events object for all events, including the pointer-missed event.

Besides that it also includes the ability to force a raycast(events fire even when mouse isn't moving) and I fixed a major oversight I came across in the pointer-enter/leave events

This should be good for more testing

alvarosabu

Amazing work @garrlker 👏🏻 we can finally merge it.

garrlker requested review from alvarosabu, Tinoooo and andretchen0 January 31, 2024 04:54

garrlker added the v4 label Jan 31, 2024

garrlker self-assigned this Jan 31, 2024

garrlker linked an issue Jan 31, 2024 that may be closed by this pull request

Pointer EventManager state #515

Open

4 tasks

garrlker added the enhancement label Jan 31, 2024

garrlker linked an issue Jan 31, 2024 that may be closed by this pull request

<TresSprite @click="() => console.log('')" /> throws error on pointer interaction #527

Open

5 tasks

garrlker mentioned this pull request Jan 31, 2024

<TresSprite @click="() => console.log('')" /> throws error on pointer interaction #527

Open

5 tasks

alvarosabu reviewed Jan 31, 2024

View reviewed changes

playground/src/pages/raycaster/Propogation.vue Outdated Show resolved Hide resolved

alvarosabu mentioned this pull request Jan 31, 2024

feat: 516 localstate for custom renderer node instances instead of userdata #522

Merged

andretchen0 requested changes Jan 31, 2024

View reviewed changes

garrlker added 3 commits February 1, 2024 12:18

Support multiple event listeners, add support for .stop event modifier

1a491da

Set stopProgation variable to false by default, whoops

580029d

andretchen0 reviewed Feb 2, 2024

View reviewed changes

garrlker force-pushed the 515-pointer-eventmanager-state branch from 977c470 to 580029d Compare February 2, 2024 04:45

garrlker changed the base branch from main to v4 February 2, 2024 04:51

garrlker added 3 commits February 1, 2024 22:57

fix typo

56776ae

fix: remove createGlobalState from useEventStore, allowing events…

de9f843

… to work while multiple TresCanvas' are being used

fix(perf): remove extraneous intersectObjects/getIntersects calls by …

d42feb3

…moving intersects into a ref that is updated on pointer-move

alvarosabu mentioned this pull request Feb 7, 2024

v4 #541

Open

13 tasks

alvarosabu mentioned this pull request Feb 20, 2024

(v4) Tests without root fail #551

Closed

5 tasks

alvarosabu added 2 commits March 7, 2024 17:37

chore(lint): fix lint issues

50dcaed

Merge branch 'v4' into 515-pointer-eventmanager-state

7783233

andretchen0 requested changes Mar 9, 2024

View reviewed changes

playground/src/pages/raycaster/Propagation.vue Outdated Show resolved Hide resolved

src/composables/useEventStore/index.ts Outdated Show resolved Hide resolved

Merge branch 'v4' into 515-pointer-eventmanager-state

a251b05

docs: new event docs

199c654

chore: fix lint

8f8ba50

alvarosabu added feature p3-significant High-priority enhancement (priority) breaking-change and removed enhancement labels Mar 28, 2024

garrlker added 3 commits April 16, 2024 00:49

Merge branch '515-pointer-eventmanager-state' of github.com:Tresjs/tr…

1677751

…es into 515-pointer-eventmanager-state

chore: lint

7aca715

garrlker requested review from andretchen0 and alvarosabu April 16, 2024 05:59

Merge branch 'v4' into 515-pointer-eventmanager-state

496cd28

alvarosabu marked this pull request as ready for review April 16, 2024 15:07

chore: fix lint issues

9981ad7

alvarosabu mentioned this pull request Apr 17, 2024

Recursive error when using a ref anywhere in the canvas when it changes on a click event handler inside a v-for #649

Open

5 tasks

alvarosabu approved these changes Apr 22, 2024

View reviewed changes

Merge branch 'v4' into 515-pointer-eventmanager-state

be99065

alvarosabu merged commit b536ab1 into v4 Apr 22, 2024
3 checks passed

alvarosabu deleted the 515-pointer-eventmanager-state branch April 22, 2024 17:38

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(events)!: pointerevents manager and state #529

feat(events)!: pointerevents manager and state #529

garrlker commented Jan 31, 2024 •

edited by alvarosabu

netlify bot commented Jan 31, 2024 •

edited

andretchen0 left a comment

andretchen0 Jan 31, 2024 •

edited

garrlker Feb 1, 2024

andretchen0 Feb 1, 2024

garrlker Feb 2, 2024

alvarosabu Feb 2, 2024

garrlker Feb 3, 2024

garrlker Mar 17, 2024

andretchen0 Feb 2, 2024 •

edited

garrlker Feb 3, 2024

garrlker Mar 17, 2024

alvarosabu commented Feb 2, 2024

andretchen0 commented Mar 9, 2024 •

edited

andretchen0 commented Mar 9, 2024 •

edited

garrlker commented Mar 17, 2024

alvarosabu commented Mar 19, 2024 •

edited

andretchen0 commented Mar 19, 2024

garrlker commented Mar 20, 2024

alvarosabu commented Mar 20, 2024 •

edited

alvarosabu commented Mar 22, 2024

andretchen0 commented Mar 22, 2024

alvarosabu commented Mar 22, 2024

andretchen0 commented Mar 22, 2024 •

edited

garrlker commented Apr 16, 2024

alvarosabu left a comment

feat(events)!: pointerevents manager and state #529

feat(events)!: pointerevents manager and state #529

Conversation

garrlker commented Jan 31, 2024 • edited by alvarosabu

Events

Event Details

Features

Chores

netlify bot commented Jan 31, 2024 • edited

✅ Deploy Preview for tresjs-docs ready!

andretchen0 left a comment

Choose a reason for hiding this comment

andretchen0 Jan 31, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

andretchen0 Feb 2, 2024 • edited

Choose a reason for hiding this comment

One click should probably cause no more than one call to a given click handler.

Currently

"Realer" world

R3F removes duplicates

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alvarosabu commented Feb 2, 2024

andretchen0 commented Mar 9, 2024 • edited

Context

andretchen0 commented Mar 9, 2024 • edited

Meta

garrlker commented Mar 17, 2024

alvarosabu commented Mar 19, 2024 • edited

andretchen0 commented Mar 19, 2024

garrlker commented Mar 20, 2024

alvarosabu commented Mar 20, 2024 • edited

alvarosabu commented Mar 22, 2024

andretchen0 commented Mar 22, 2024

alvarosabu commented Mar 22, 2024

andretchen0 commented Mar 22, 2024 • edited

garrlker commented Apr 16, 2024

alvarosabu left a comment

Choose a reason for hiding this comment

garrlker commented Jan 31, 2024 •

edited by alvarosabu

netlify bot commented Jan 31, 2024 •

edited

andretchen0 Jan 31, 2024 •

edited

andretchen0 Feb 2, 2024 •

edited

andretchen0 commented Mar 9, 2024 •

edited

andretchen0 commented Mar 9, 2024 •

edited

alvarosabu commented Mar 19, 2024 •

edited

alvarosabu commented Mar 20, 2024 •

edited

andretchen0 commented Mar 22, 2024 •

edited