lag (or misapplied prediction?) in the HTC Vive HMD tracking

[jzitelli]

With the HTC Vive, I observe a lag between movement of the chaperone bounds and the rendered WebVR scene when the HMD is tracking. It looks like the chaperone visual is lagging behind. I tried to capture a video of the effect, recorded from the SteamVR mirror display, but the lag doesn't show up in the mirror display.

[jzitelli]

Sorry, the tracking of the scene clearly lags behind the chaperone, not the other way around... so I guess this could just be summed up as lag (or misapplied prediction?) in the HTC Vive HMD tracking, I've changed the issue title to reflect that.

[sminnee]

This might be a duplicate of #10?

[caseyyee]

I did some testing and I don't believe that the issue is due to latency.

As you rotate the headset, you will see that the scene will shift out of alignment with the chaperone. Interestingly, when the headset is translated, the tracking error does not occur. So it seems that it is something to do with how the rotation is being rendered or the rotation values in the pose itself.

The issue is most notable against the chaperone, but it is visible even inside the chaperone bounds. It's just not as obvious if you are not looking for it.

[sminnee]

@caseyyee in your tests, does the scene return to line up correctly with the chaperone once you stop moving? Or is it permanently misaligned?

My experience has certainly been that it's a temporary error, so it's related to delayed update of some kind, but it's interesting that it impacts orientation more strongly than position.

[caseyyee]

It will stay misaligned if you stop moving the headset.

There could be some latency, but I'm not sure that it's related with this issue.

[toji]

Okay, that's a new one. I've seen that the chaperone can be out of sync with the WebVR scene before, but I don't recall ever seeing it become permanently misaligned. When this happens is it possible that the lighthouse units have become occluded?

[capnmidnight]

That sort of misalignment I've seen in my own apps. It's sort of like the
headset position is scaled by a factor of 2 or something.

I had fixed it by looking closely into your Room Scale sample. I believe it
fixed when I applied the sittingToStanding transform, but that doesn't make
a lot of sense because that transform is only a translation in the Y axis,
IIRC. I've recently seen it again, but I've been in the middle of rewriting
my locomotion, so i just assumed I had broken whatever I fixed before.
On Jul 14, 2016 6:59 PM, "Brandon Jones" notifications@github.com wrote:

Okay, that's a new one. I've seen that the chaperone can be out of sync
with the WebVR scene before, but I don't recall ever seeing it become
permanently misaligned. When this happens is it possible that the
lighthouse units have become occluded?

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#73 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AASMPq6cJLWxUPNpWAzVUy8LKDsNIk3Uks5qVr9UgaJpZM4IznLd
.

[toji]

FYI: If you're using a Vive the sittingToStandingTransform may also involve rotation and translation.

[jzitelli]

@caseyyee for me the chaperone does not stay misaligned, I see what @sminnee describes... I believe (just based on subjective observation, and assuming that the chaperone does not exhibit any latency since it's automatically composited by openvr) this is just an indication of overall latency (issue #10).

Re: chaperone behavior under translation vs. rotation, in my experience rotation tends to exhibit subjective effects of latency much more clearly than translation, since even a slow rotation of your head can result in a fast change in visuals (proportional to the distance away of rendered objects) when projected to the image plane, which is hard to match by just moving your head left/right/up/down, and the near plane clips off the geometry where it would be most apparent in such case.

[caseyyee]

When you rotate your head, the registration of the chaperone doesn't match the scene. The mismatch remains consistent and does not change over time and is directly related to the rotation of your head. It doesn't have anything to do with latency as it doesn't matter how fast or slow you are moving your head.

It's only subjective as far as if you can actually perceive the error or not. For me, it's quite apparent and honestly makes me a bit sick (i'm particularly prone to sim-sickness), since what effectively is happening is that the headset orientation is not matching the rendering of the scene.

I'll work on trying to capture some video of this.

[capnmidnight]

I'm consistently seeing this issue in every Vive-supporting WebVR demo I've found. It makes it seem like the pose that is being reported through the API is three or four frames behind the actual headset. If you open the system menu, you can more easily see the steadiness of the home room with the latency of the WebVR session blended into it.

[bnolan]

Yeah I've noticed this too. I guess it's either the webgl pipeline takes 3 frames to get the GPU, or somehow the function that returns the pose is returning something 3 frames behind. I assumed it's more likely the former, since it goes through a few compositing stages. It makes the vive pretty uncomfortable.

[Spaxe]

I noticed something similar but not quite: the tracking seems to be off between the chaperone and the floaty objects. When I rotate my head around, the two moves in different parallax, which is disillusioning. HTC Vive consumer version.

This issue is not present in samples that don't require room scale.

[capnmidnight]

I have yet to try setting up my Vive in "standing only" setup.

I finally got back in to the office to try it out again with the latest (as of September 8th) builds and Firefox does not display anywhere near the same issue. It might be showing a little bit, but if it is, it's well within acceptable range. It's really bad in Chromium. Makes it feel like walking around during an earthquake.

Framerate is fine, and actions within the scene do not lag, so it's not rendering latency. It's just the orientation.

[ollyhayes]

I'm also seeing this latency in Chromium.

You can view it in this page: https://olly.fr.to/vr.html. Position the box into the corner of the chaperone with the arrow keys and then just stand on the other side of the room. In Chromium you only need to wiggle your head very slightly to get the box to jump to either side of the Chaperone, where as in Firefox it looks completely fixed.

Like jzitelli says in his first comment, when viewing this in the headset mirror the box and the chaperone move as one.

Is this affecting all vive/chromium users or is it dependent on card/drivers etc?

[Macil]

I think there might be multiple issues in this thread? One about latency, and one about rotation (as described by @caseyyee). I'm seeing the same issue described by @caseyyee always. As I rotate the direction I'm looking, the entire scene shifts. The effect is very easy to see when the chaperone bounds are visible, because they stay in place, making it obvious that everything else is shifting. Even without the chaperone to compare to, the effect is nauseating.

Here's some screenshots. It's of the roomscale sample at webvr.info. I take one screenshot (press the system button and then both triggers while holding it) looking right at the chaperone edge (it appears to be a couple inches from the floor cube), and then I turn my head almost 90-degrees left. You can see the chaperone edge now nearly meets up with the floor cube on the right side.

This issue doesn't seem to have anything to do with latency. If I walk around the room while doing my best to keep the headset rotation the same, then the floor-cube stays still relative to the chaperone bound. The position of the scene seems to be tied to the specific angle the headset is pointed at: rotating the headset and returning to the same angle puts the scene back into the same place.

Firefox Nightly's webvr seems to have the same bug. The samples on webvr.info and Mozilla's A-frame samples show the issue in both Chrome and Firefox. No other application or game I've used has shown this issue.

[klausw]

Is WebVR using the GetEyeToHeadTransform matrices from OpenVR? If those include 3D offsets from the HMD origin, it would be wrong to just apply a 1D +/- IPD adjustment. The symptoms seem to match a wrong head rotation origin.

[caseyyee]

I just tested this with the Oculus Rift + Touch controllers setup with guardian and am seeing the exact same issue where the scene shifts with headset rotation (as per my earlier comments).

[klausw]

Here's a quick test what the getEyeToHeadTransform matrices look like for my Vive:

[[1.0, 0.0, 0.0, -0.033649999648332596],
 [0.0, 1.0, 0.0, 0.0],
 [0.0, 0.0, 1.0, 0.014999999664723873]]

[[1.0, 0.0, 0.0, 0.033649999648332596],
 [0.0, 1.0, 0.0, 0.0],
 [0.0, 0.0, 1.0, 0.014999999664723873]]

These matrices have both an X offset (+/- half IPD), and also a nonzero Z offset of 1.5cm.

Assuming https://chromium.googlesource.com/experimental/chromium/src/+/refs/wip/bajones/webvr_1/device/vr/openvr/open_vr_device.cc#104 is the current version, the WebVR driver is just using +/- half IPD without a Z offset.

I think it's definitely worth trying using the full adjustment. This may not be the only issue, but a 1.5cm Z offset error would be likely to be noticeable.

Here's the test code, based on a Python openvr library for convenience:

import openvr
openvr.init(openvr.VRApplication_Scene)
vr_system = openvr.init(openvr.VRApplication_Scene)
print vr_system.getEyeToHeadTransform(openvr.Eye_Left)
print vr_system.getEyeToHeadTransform(openvr.Eye_Right)
openvr.shutdown()

[klausw]

So looks like there's two problems after all:

• normal-speed or fast movement has a noticeable lag until the virtual world moves to its correct position. You can see world movement continue after you stop moving until it's caught up.

• the eye position is in slightly the wrong place. If you move your head very slowly (so that you're not affected by the lag issue), you can see that there's a persistent offset that depends only on current head orientation and doesn't go away if you stop moving or do translational movement.

I've hacked in a quick workaround to correct the eye Z offset on the Javascript side here: https://klausw.github.io/webvr.info/samples/05-room-scale.html . This fixes the offset issue, but the lag still remains :-/

[huningxin]

@klausw , the Feb 1 build seems fix this issue. Are you able to share the root cause and fix here? Thanks!

[sjpt]

For me, the asynchronous reprojection works well with Chrome/Vive if I use webgl2, but terrible if I use regular webgl. This is using the Dec Chrome build as I have other issues with the Feb 1 build.

[natbro]

natb from valve here. the eye position does seem corrected in today's Chromium build on Vive (I don't know if it fixed earlier, those wouldn't work for me). the lag is a known issue with the way frames propagate through chromium's rendering path and get submitted to the compositor. today's build seems a frame better than in Oct/Nov, but there are still 2-3, maybe 4, frames of delay from the pose that frames are rendered with vs the frame they go out with. you see this when you line an eye up with a chaperone boundary and a piece of content - as you move your head back and forth the chaperone bound will stay steady but the content will wiggle.

[toji]

Closing all bugs in this issue tracker.

This repo was created to track issues in the experimental builds WebVR Chromium builds, which are now deprecated. Chrome Canary for Windows now has much more secure (and hopefully more performant) support for WebVR behind a flag, and Android has had WebVR support as an Origin Trial and behind a flag for a while now.

If this is a performance or correctness bug and you suspect it's still happening, please test against the latest Canary build of Chrome to verify and then file a bug at https://crbug.com. If this is an issue with the API, please review the latest WebXR explainer to see if it's been resolved and file a bug there if not.

Thanks for your interest in VR on the web! We've got an exciting year ahead of us!
--Brandon