Backend.OpenXR.PollEvent for spatial anchors?

[jackdaus]

Hey!

So I was taking a look at implementing the OpenXR extension XR_FB_spatial_entity in StereoKit's C# layer (https://github.com/jackdaus/FBSpatialEntityDotNet). I seem to have hooked up all the function pointers, but it looks like Meta took an async approach with their spatial anchors. In order to access the results of these function calls, we need to tap into the xrPollEvent function.

Maybe it makes sense to make the xrPollEvent method available in SK's Backend.OpenXR? Not sure how much work it would be, or if it really makes sense... I see in the OpenXR docs it says of xrPollEvent,

The event (if present) is unilaterally removed from the queue if a valid XrInstance is provide

So I imagine this would cause issues if both StereoKit and the client app call xrPollEvent. Since in that case, one process would consume an event from the queue and the other process would never pick up on the that event.

Anyways, I see you made a lot of progress on SK's spatial anchor API! So maybe at this point it makes the most sense to just implement the FB spatial anchors in the StereoKitC layer and avoid the complexities associated with surfacing xrPollEvent in the Backend.OpenXR API.

(If you're not planning on doing the FB anchor implementation soon, I could try to take a stab at it! My work might not be up to par to merge into the SK repo, but it could at least be a good learning experience for me 😊)

[maluoi]

I've kinda had it in the back of my mind that data from xrPollEvent may need to be exposed at some point, so this is likely something I'll want to do even independent of the anchor stuff. Thanks for bringing it up! This will likely come in the form of a subscription callback that will be invoked from within StereoKit's call to xrPollEvent.

Related to FB's spatial anchors, so far I've mostly been looking at the MSFT anchors! But I am planning on getting to the FB ones as well. I'm designing an API that will hopefully encompass most of the anchor-like concepts in OpenXR, but that may mean it won't include some more fringe vendor-specific features. I'd love to know what functionality of the FB anchors you're interested in, and how you might be planning to use them! Details like that may help inform where exactly the API goes, and what it contains in the end :)

I am hoping to get the anchor API into this release, but it is turning out to be one of the more complicated public interfaces I've designed. And I don't love shipping complicated APIs for people :D So we'll see.

[jackdaus]

Yeah sure! So how I'm planning to use these anchors: I'm working on an augmented reality inventory app. The idea is that the app will be a system where users can label items in their real world environment. The user can place an anchor at the location of a real world item and then give the anchor a label (plus some additional metadata). After the anchors have been placed in the enviornment, the user can query the system to get directions to the items. The directions would appear as virtual arrows or a path. I'm imagining the use case would be something like a guide for a warehouse employee or maybe even a guest at an Airbnb. (This is all for a university course assignment, so probably not actually going to be used like this anytime soon.)

As a V1, I would like to use the anchors just to simply have the augmented labels persist at the correct real world location across user sessions. (I'm not yet sure if I'd need 1 anchor for each item in the inventory, or maybe I'll just create 1 anchor that will be shared by a bunch of Inventory Items. I need to look into that further.)

As a V2, it would be really cool to tap into the room layout info that the Quest might have available. This way, my app could take into account walls and doors during navigation. This looks like something that might be available more in the XR_FB_scene extension, but I haven't looked into that one much yet.

[jackdaus]

Would it be cool if I took a stab at surfacing xrPollEvent and made a PR? If it's something you'd rather handle, no prob! Just wanted to offer :)

[maluoi]

If you want to take a stab at it, go for it, yeah! I think the important detail is exposing it via an API that doesn't require openxr.h, as long as it does that, should be great :)

[jackdaus]

Ok cool, I'll give it a shot and keep that in mind!