[Bug Report] VR Experience broken with IsaacLab / IsaacSim 4.0 and 4.1

[fayezsalka]

When enabling VR Experience in Isaac Sim, the experience end up broken on both Windows and Linux, with SteamVR and OpenXR. Rendering works fine but the controllers and device position are tracked incorrectly. Same exact setup works ok in Isaac Sim 2023.1.1

[omni.kit.xr.core.plugin] ERROR getting world matrix from RT xformable

This makes it impossible to use IsaacLab / IsaacSim for virtual robot teleoperation.

[Mayankm96]

This issue seems related to Isaac Sim rather than Isaac Lab itself. I would suggest reporting it on the NVIDIA Developer Forums for Isaac Sim so that the right team can help you. You can do that here: https://forums.developer.nvidia.com/c/omniverse/simulation

[rthaker01]

Hi @fayezsalka - You should be able to use VR Experience with latest Isaac Sim 4.2/Isaac Lab 1.2 release. Please try and confirm.

[scode-cmd]

Hi @fayezsalka - You should be able to use VR Experience with latest Isaac Sim 4.2/Isaac Lab 1.2 release. Please try and confirm.

Yep. Can confirm VR Experience now works nicely with Isaac Sim 4.2. I was able to run the Humanoid demo at 90hz on my Meta Quest 3. Pretty fun!

Only issue is that teleportation movement mechanic seems to be jittery and no way to disable it. The teleportation mechanic takes over the joystick of the quest controllers, which we rather need for other things like controlling our robot itself. For now the only solution I found was modifying the metaquest extension in cache and removing all controller bindings so we can have our own custom bindings with the quest’s joystick.

[parilo]

Hey, is there any documentation on how to use VR in Isaac Sim / Isaac Lab? Or any suggestion how to run it?

[sasasqt]

Hey, is there any documentation on how to use VR in Isaac Sim / Isaac Lab? Or any suggestion how to run it?

ig vr exprience is this plugin: omni.kit.xr.profile.vr

Virtual Reality Extension [omni.kit.xr.profile.vr]

This extension configures the xr subsystem for a virtual reality experience.
The extension supports virtual reality either directly using SteamVR or streaming with CloudXR.

[Bailey-24]

could you please share your script, how to teleop the robot using VR (meta quest 3)?

@Bailey-24 any updates on this? Did you get the file, did you get the teleop working?

[attila-lingenieur]

Any update ? Has anyone managed to make Meta Quest 3 work in IsaacLab? ( and share solution please)